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1.0  Introduction 


This  document  reports  on  research  into  applying  genetic 
algorithms  to  function  decomposition  in  Pattern  Theory.  The 
Mission  Avionics  Division  of  the  Avionics  Directorate  at 
Wright  Laboratories  (WL/AART)  is  developing  an  engineering 
design  theory  for  algorithms,  based  on  the  structure  or 
"pattern"  of  a  function  called  Pattern  Theoi^.  The  term 
function  refers  to  the  traditional  mathematical  function 
associating  inputs  with  specific  outputs.  Pattern  Theory 
has  relevance  to  a  number  of  disciplines  to  which  these 
functions  may  relate,  including  Pattern  Recognition, 
Artificial  Intelligence,  data  compression,  cryptography,  and 
Switching  Theory.  Decomposed  Function  Cardinality  (DFC) 
refers  to  the  complexity  of  a  function.  The  DFC  is  found  by 
reducing  the  function  to  its  simplest  representation. 
Function  decomposition  optimizes  the  function  with  respect 
to  the  DFC  using  a  systematic  approach.  The  measure  of  DFC 
has  been  shown  to  correlate  well  with  traditional  measures 
of  computational  complexity  such  as  program  length,  time  and 
circuit  size  complexity.  The  ability  to  reduce  functions  to 
their  simplest  representation  in  the  most  optimal  way  can 
simplify  designs,  reduce  computational  time,  and  lower 
costs. 

Function  decomposition  groups  or  partitions  input 
variables  for  a  function  into  smaller  functions.  Functions 
can  be  represented  by  two  dimensional  arrays,  by  treating 
the  input  variables  as  labels  for  the  rows  and  columns  of 
the  arrays.  These  arrays  are  called  partition  matrices  and 
several  may  be  generated  for  each  function  by  selecting 
different  variables  for  the  rows  or  columns.  This  selection 
procedure  is  called  partition  selection  and  can  be  used  to 
reduce  functions  to  their  optimal  decomposition.  Among  the 
partition  selection  approaches  that  can  be  used  to  obtain 
the  DFC  are  random  or  chance  procedures  and  correlation  of 
the  values  of  input  variables  and  values  of  the  function. 
Mission  Avionics'  personnel  have  found  a  more  promising 
technique  by  using  genetic  algorithms.  These  algorithms  use 
search  procedures  modeled  after  the  biological  process  of 
natural  selection  and  genetics.  In  the  application  of 
genetic  algorithms  to  function  decomposition,  information 
learned  from  previous  decompositions  guides  further 
decompositions.  Genetic  algorithms  tend  to  reduce  the 
search  space  and  guide  the  search  more  effectively  than 
other  methods. 

Section  2  of  this  report  gives  some  background  on 
Pattern  Theory  and  its  development.  Also  Section  2  more 
clearly  defines  Function  Decomposition  and  DFC  and  gives 
some  examples.  Partition  selection  in  the  decomposition 
process  is  described,  including  possible  techniques  for 
optimizing  the  decomposition.  Background  is  also  given  on 
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Genetic  Algorithms,  including  the  definition  of  heuristics 
and  parameters  used  in  genetic  algorithms  and  a  comparison 
with  their  biological  equivalents. 

The  third  section  describes  specifically  the 
application  of  genetic  algorithms  to  partition  selection  in 
the  function  decomposition  process.  This  includes  the 
representation  of  the  problem,  the  techniques  for  performing 
a  structured  search,  and  the  evaluation  of  individual 
solutions.  A  description  is  given  of  the  various  parameters 
that  were  varied  in  this  program  and  the  different 
heuristics  which  were  applied.  A  comparison  is  made  between 
the  two  primary  genetic  algorithm  approaches  taken  to 
perform  partition  selection.  One  approach  is  referred  to  as 
the  baseline  genetic  algorithm,  following  more  standard 
genetic  algorithm  procedures,  and  the  other  is  referred  to 
as  the  structured  approach.  The  various  approaches  chosen 
to  implement  these  genetic  algorithms  to  optimize  function 
decomposition  are  detailed,  particularly  those  that  deviate 
from  classic  genetic  algorithm  procedures. 

Section  4  describes  experiments  performed  to  evaluate 
the  application  of  the  baseline  and  structured  search 
algorithms  to  function  decomposition.  Since  computational 
time  would  have  severely  limited  this  program,  a  number  of 
decompositions  were  simulated  without  exhaustive  computer 
runs.  The  process  used  for  this  simulation  is  described.  A 
comparison  is  made  for  determining  the  DFC  between  simulated 
decompositions  and  another  estimation  procedure. 

The  document  concludes  by  offering  conclusions  from  the 
e^eriments  and  then  offers  recommendations  for  the 
direction  of  future  work. 
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2.0  Background 


2.1  Pattern  Theory 

Pattern  Theory  describes  an  algorithm  design  paradigm 
being  developed  at  Wright  Laboratory.  It  attempts  to  help 
automate  the  algorithm  design  process.  Pattern  Theory  uses 
a  function  decomposition  algorithm  to  reduce  complicated 
functions  to  their  more  simple  components.  By  using 
function  decomposition  it  is  possible  to  design  an  algorithm 
that  implements  a  function  given  only  a  partial  list  of 
inputs  and  outputs.  Functions  decompose  in  a  number  of 
ways.  The  function  decomposition  algorithm  conducts  an 
iterative  and  recursive  search  until  the  simplest 
decomposition  is  found.  This  search  automatically  designs  an 
algorithm  for  reconstructing  the  original  function  using  the 
simplest  decomposition.  It  simultaneously  provides  a 
measure  of  the  intrinsic  complexity  of  the  original 
function.  This  measure  of  complexity  is  called  decomposed 
function  cardinality  or  DFC.t^J 

The  idea  of  using  the  simplest  representation  is 
alternately  known  as  the  principle  of  parsimony  or  Occam's 
razor.  The  belief  that  the  simplest  explanation  or 
representation  is  best  has  been  a  basic  tenet  of  the 
philosophy  of  science.  It  may  be  the  only  thing  that  allows 
us  to  think,  do  math  or  science,  or  make  any  predictions 
about  the  world  at  all. 

Function  decomposition  as  used  in  pattern  theory 
provides  a  potentially  powerful  technique  to  automate 
algorithm  design.  This  has  a  myriad  of  possible 
applications  for  automated  code  generation,  circuit  design, 
cryptography,  data  compression,  image  processing  and  machine 
learning.  This  technique  has  been  demonstrated  in  more  than 
one  thousand  experiments  with  impressive  results.  These 
experiments  used  small  binary  functions  of  less  than  ten 
input  variables. 

A  problem  exists  for  functions  with  more  input 
variables.  The  search  space  increases  exponentially  with 
the  number  of  input  variables.  Consequently  the  computer 
time  required  to  find  the  DFC  becomes  prohibitive.  At 
present,  there  is  no  direct  method  that  guarantees  finding 
the  DFC  without  exhaustive  search.  There  are  a  number  of 
methods  that  can  potentially  solve  this  search  problem. 

Some  promising  techniques  that  limit  the  search  space  or 
guide  the  search  are  being  investigated.  Among  these  are 
genetic  algorithms. 

Considerable  research  has  recently  been  performed  in 
the  area  of  genetic  algorithms.  They  can  be  very  useful  for 
combinatorial  hard  optimization  problems.  This  is  the  type 
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of  problem  we  are  facing  in  function  decomposition.  Using 
an  exhaustive  search  to  find  a  solution  is  really  no  better 
than  chance.  Genetic  algorithms  purport  to  be  an 
improvement  over  chance.  The  amount  of  improvement  depends 
on  the  nature  of  the  problem  and  the  adjustment  of 
parameters  and  heuristics  in  the  genetic  algorithm.  This 
report  documents  the  potential  for  using  genetic  algorithms 
for  function  decomposition. 

An  introduction  to  function  decomposition  and  DFC  is 
presented  in  Pattern  Theory;  An  Engineering  Paradigm  for 
Algorithm  Design  m.  The  work  reported  here  is  part  of 
Pattern  Theory  II.  A  summary  of  this  expanded  work  and 
other  results  will  be  published  in  a  later  technical  report. 
The  following  discussion  of  function  decomposition  and 
simple  example  are  presented  here  for  the  sake  of 
completeness  and  should  aid  the  reader  who  is  unfamiliar 
with  the  decomposition  process.  It  also  defines  terms  used 
in  the  application  of  genetic  algorithms  to  function 
decomposition . 

2.1.1  Function  Decomposition  (The  Simple  Explanation) 

Function  decomposition  can  apply  to  any  type  of  finite 
function:  binary,  numerical  symbolic,  etc.  For  the  rest  of 
this  discussion  and  the  following  example,  we  will 
concentrate  on  binary  functions.  A  binary  function  may  be 
represented  as  a  string,  one  dimensional  array  or  lookup 
table.  Table  2.1  shows  a  table  representation  of  a  binary 
function.  This  table  will  be  referenced  again  later  in  an 
example  of  the  function  decomposition  process  illustrating 
table  representations  of  functions  and  subfunctions.  For 
this  explanation  of  the  process,  simple  diagrams  will  be 
used. 


Table  2.1.  A  Binary  Function  Expressed  as  a  String. 


a 

b 

c 

d 

e 

f 


000000000000000000000000000000001 1111111111111111111111111111111 
00000000000000001 111111111111111 00000000000000001 111111111111111 
000000001 1 1 1 1 1 1 1000000001 1111111 000000001 11111 11000000001 1111111 
00001 1 1 1 00001 1 1 1 00001 1 1 1 0000 1 1 1 1 00001 1 1 1 00001 1 1 1 00001 1 1 1 00001 1 1 1 
0011001100110011001100110011001100110011001100110011001100110011 
0101010101010101010101010101010101010101010101010101010101010101 


F  1  0000011000001000011000001000000001100000100000000000000000000000 


In  general  if  there  are  N  input  variables  to  a  binary 
function  then  the  number  of  elements  in  the  table 
representation  is  2^.  The  number  of  elements  is  also 
referred  to  as  the  cardinality  of  the  function.  Such  a 
function  can  be  diagrammed  as  shown  in  Figure  2.1. 
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Figure  2.1.  Diagram  of  a  Function  with  N  Input  Variables 

and  Table  Size  of  2N. 


Some  functions  can  be  represented  as  a  composition  of 
smaller  functions  or  subfunctions.  Suppose  the  function  in 
Figure  2.1  can  be  split  into  two  functions.  M  of  the  input 
variables  form  the  input  to  Function  A.  Now  the  output  of 
Function  A  and  the  remaining  N-M+1  inputs  (Function  B)  form 
the  input  to  Function  F.  Two  new  tables  are  formed  in  the 
alternative  representation.  The  sum  of  the  cardinality  of 
the  new  functions  is  2**  +  2  (N-M+i)  ^  decomposition  is 

diagrammed  in  Figure  2.2. 


Figure  2.2.  Function  F  Decomposes  to  a  Table  Size  of  2M  + 

2 (N-M+1) . 


A  good  decomposition  is  one  that  simplifies  the 
representation  and  offers  a  cost  savings,  i.e.,  if  the  sum 
of  the  new  cardinality  is  less  than  the  original  cardinality 
2M  +  2 (N-M+1)  <  2^.  Suppose  that  Function  A  further 
decomposes  into  Functions  C  and  D  (see  Figure  2.3).  Figure 
2.4  shows  the  case  where  Function  B  decomposes.  If  both 
Functions  A  and  B  decomposed  the  case  would  appear  as  shown 
in  Figure  2.5.  The  sum  of  the  new  cardinality  would  be 
2?  +  2(M-P+1)  +  2Q  +  2(N-M-Q+2). 
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Figure  2.3.  Function  A  Decomposes  to  a  Table  Size  of  2P  + 

2 (M-P+1) . 


Figure  2.4.  Function  B  Decomposes  to  a  Table  Size  of  2Q  + 

2(N-M-Q+2) . 


Figure  2.5.  Both  Functions  A  and  B  Decompose. 


There  is  one  final  case  to  consider.  Sometimes  a 
nxamber  of  variables  can  be  removed,  but  more  than  one 


function  is  produced.  In  this  case  two  or  more  (m  for 
example)  tables  of  size  M  are  produced  (see  Figure  2.6). 
The  sum  of  the  new  cardinality  is  m*(2^)  +  2  (N“M+m)  ^ 
long  as  m*(2®®)  +  2(N“M+in)  ig  less  than  2^  this  is  a  good 
decomposition . 


Figure  2.6.  Decomposition  Producing  Two  or  More  Functions 

of  Table  Size  M. 


2.1.2  Function  Decomposition  Example 

The  following  is  an  illustration  of  the  decomposition 
process  using  a  binary  function  represented  by  the  string  or 
table  shown  previously  in  Table  2.1.  N  or  the  number  of 
input  variables  is  6  in  this  case  and  the  cardinality  is  2*^ 
or  64.  The  binary  input  variables  are  a,  b,  c,  d,  e  and  f. 

F  is  the  function. 

This  function  can  also  be  represented  by  two 
dimensional  arrays.  This  is  accomplished  by  treating  the 
input  variables  as  labels  or  indices  for  the  rows  and 
columns  of  the  arrays.  These  arrays  are  referred  to  as 
partition  matrices.  Several  possible  partition  matrices  of 
F  are  shown  in  Tables  2.2,  2.3  and  2.4. 

Partition  Matrices  of  the  Function  F. 


Table  2.2.  Table  2.3.  Table  2.4. 


a 

00001111 

a 

00001111 

a 

0000000011111111 

b 

00110011 

b 

00110011 

b 

0000111100001111 

c 

01010101 

d 

01010101 

c 

0011001100110011 

def 

cef 

d 

0101010101010101 

000 

00010100 

000 

00000000 

ef 

001 

00101000 

001 

01101000 

00 

0001001000100000 

010 

00101000 

010 

01101000 

01 

0100100010000000 

oil 

00000000 

oil 

00000000 

10 

0100100010000000 

100 

01000000 

100 

01101000 

11 

0000000000000000 

101 

10000000 

101 

00000000 

110 

10000000 

110 

00000000 

111 

00000000 

111 

00000000 
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Different  partition  matrices  are  generated  by  selecting 
different  input  variables  to  label  the  rows  and  columns  of 
the  array.  The  selection  of  variables  to  use  for  row  or 
col^Imn  labels  is  called  partition  selection. 

After  examining  the  partition  matrices  in  Tables  2.2, 
2.3,  and  2.4,  one  notices  that  in  each  partition  matrix  some 
of  the  columns  are  identical.  Since  some  of  the  columns  are 
identical,  the  number  of  unique  columns  in  a  partition 
matrix  is  less  than  the  total  number  of  columns.  The  nximber 
of  unique  columns  in  a  partition  matrix  is  called  column 
multiplicity.  Column  multiplicity  is  important,  since  it 
can  be  used  to  determine  if  a  function  will  decompose  at  a 
cost  savings.  If  the  number  of  unique  columns  can  be 
represented  by  fewer  binary  variables  than  is  required  to 
represent  the  number  of  columns  in  the  original  partition 
matrix,  then  the  function  usually  can  be  broken  down  into 
several  smaller  functions.  A  second  condition  for 
decomposition  requires  that  the  ntamber  of  binary  variables 
needed  to  represent  the  number  of  unique  columns  is  less 
than  the  number  of  rows  in  the  partition  matrix. 

Consider  the  partition  matrix  shown  in  Table  2.2.  It 
has  five  unique  colvimns.  Since  three  binary  variables  are 
required  to  express  represent  decimal  five,  this  partition 
matrix  cannot  be  decomposed  at  a  cost  savings. 

The  partition  matrix  in  Table  2.3  has  only  two  unique 
colximns.  This  requires  one  binary  variable.  A  new 
component  function  g  is  created  by  assigning  binary  labels 
to  the  uni^e  columns  as  shown  in  Table  2.5.  Notice  that  g 
is  a  function  of  column  variables.  The  original  function 
can  now  be  expressed  as  a  function  of  the  row  variables  and 
g  (Table  2.6). 
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Table  2.5.  New  Component  Function  g  Created  by  Assigning 
Binary  Labels  to  the  Unique  Columns  in  This  Partition 

matrix. 


a 

00001111 

b 

00110011 

d 

01010101 

cef 

000 

00000000 

001 

01101000 

010 

01101000 

oil 

00000000 

100 

01101000 

101 

00000000 

110 

00000000 

111 

00000000 

g 

01101000 

Table  2.6.  Original  Function  Expressed  as  Function  of  Row 

Variables  and  g. 


a 

00001111 

c 

0000000011111111 

b 

00110011 

e 

0000111100001111 

d 

01010101 

f 

0011001100110011 

g 

01101000 

g 

0101010101010101 

F 

0001010001000000 

The  sum  of  the  cardinality  to  express  g  and  F 
(23+2^=24)  is  already  less  than  the  cardinality  to  express 
the  original  function  (2^=64) .  However,  we  are  not  done 
yet.  We  try  to  decompose  F  and  g  again.  The  function  g 
cannot  be  decomposed  any  further,  but  F  can  be  decomposed 
again.  This  time  we  select  c,  e,  and  f  as  column  variables 
and  arrive  at  another  intermediate  function  h  (Table  2.7). 

Table  2.7.  Intermediate  Function  h  Obtained  by  Decomposing 

F  Again. 


c 

00001111 

e 

00110011 

f 

01010101 

g 

0 

00000000 

1 

01101000 

h 

01101000 
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Finally,  we  express  F  as  a  function  of  g  and  h.  The 
intermediate  function  g  and  h  are  functions  of  the  input 
variables.  Table  2.8  shows  the  final  decomposition.  Neither 
F,  g  nor  h  can  be  decomposed  any  further.  The  sum  of 
cardinality  is  20  {^2^ +2^ +2^)  instead  of  64. 

Table  2.8.  F  Expressed  as  a  Function  of  g  and  h. 


a 

00001111 

c 

00001111 

g 

0011 

b 

00110011 

e 

00110011 

h 

0101 

d 

01010101 

f 

01010101 

F 

0001 

g 

01101000 

h 

01101000 

What  if  there  are  more  than  two  unique  columns  as  in 
the  partition  in  Table  2.4,  then  functions  g  and  h  would  be 
created  (see  Table  2.9).  Again  these  new  functions  are  only 
a  function  of  column  variables.  The  functions  g  and  h  are 
decomposed  again  in  Table  2.10  and  the  original  function  can 
now  be  expressed  as  a  function  of  the  row  variables  and  the 
new  functions. 

Table  2.9.  Functions  g  and  h  Created  From  the  Partition  in 

Table  2.4. 


a 

0000000011111111 

b 

0000111100001111 

c 

0011001100110011 

d 

0101010101010101 

ef 

00 

0001001000100000 

01 

0100100010000000 

10 

0100100010000000 

11 

0000000000000000 

g 

0100100010000000 

h 

0001001000100000 

Table  2.10.  Original  Function  Expressed  as  Function  of  Row 

Variables  and  New  Functions. 


a 

b 


c 

d 


0000000011111111 

0000111100001111 

0011001100110011 

0101010101010101 


g 


0100100010000000 


a 

0000000011111111 

b 

0000111100001111 

c 

0011001100110011 

d 

0101010101010101 

h 

0001001000100000 
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Note  the  Xs  in  the  final  function  shown  in  Table  2.11. 
These  represent  "don't  care"  conditions.  There  is  no 
combination  of  input  variables  that  can  produce  any  output 
here,  since  the  intermediate  functions  are  never  "one" 
simultaneously.  The  sum  of  cardinality  for  the 
decomposition  at  this  level  is  40  (2^+22+22+22+2^) . 


Table  2.11.  Final  Decomposition  of  Partion  in  Table  2.4. 


a 

00001111 

b 

00110011 

d 

01010101 

i 

01101000 

a 

00001111 

b 

00110011 

d 

01010101 

j 

01101000 

c 

0011 

i 

0101 

g 

0100 

c 

0011 

j 

0101 

h 

0001 

e 

f 

g 

h 

F 


0000000011111111 

0000111100001111 

0011001100110011 

0101010101010101 

OIOXOOIXOOIXOOOX 


Notice  that  i  and  j  are  identical  functions  of  a,  b  and 
d.  They  can  be  shared  in  the  final  decomposition  as  Figure 
2.7  shows.  The  final  sum  of  cardinality  for  this 
decomposition  is,  therefore,  32  (2^+22+22+2^) .  This  point 
is  important  because  it  will  later  be  used  to  calculate  the 
cost  of  decompositions. 


Figure  2.7.  Diagram  of  Final  Decomposition  of  Partition  in 
Table  2.4,  Showing  the  inputs  a,  b,  and  d  can  be  shared. 


Summing  the  cardinality  of  the  new  functions  (20  in 
Table  2.8  and  32  in  Table  2.11  with  a,  b  and  d  shared)  shows 
that  both  are  less  than  the  cardinality  of  the  original 
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fvmctions  (64  in  both  Tables  2.3  and  2.4).  The  sum  of 
cardinality  of  the  new  functions  generated  in  Table  2.8  is 
smallest.  This  is  an  example  function  decomposition 
process.  It  is  applied  for  all  partition  matrices  and  again 
to  each  of  the  new  functions  generated  until  the 
representation  with  the  smallest  sum  of  cardinality  is 
found.  The  number  of  elements  (cardinality)  needed  for  this 
minimal  representation  is  the  measure  of  complexity  used  by 
pattern  theory.  It  is  called  decomposed  function 
cardinality  or  DFC. 


2.2  The  Combinatorial  Problem 

The  number  of  possible  non  trivial  partition  matrices 
for  a  given  binary  function  is  the  cardinality  minus  two. 

For  the  example,  there  were  62  partition  matrices  for  the 
function  in  Section  2.1.2.  Only  a  small  number  of  the 
possible  partition  matrices  were  shown  to  illustrate  a  few 
points  about  the  decomposition  process.  Table  2.12  shows 
the  minimum  sum  of  cardinality  that  can  result  from  all 
possible  initial  partition  matrices  of  the  original 
function.  Note  that  there  are  several  partition  selections 
that  lead  to  the  DFC  (20)  and  many  others  that  lead  to  some 
intermediate  value  between  the  original  cardinality  and  DFC. 
The  importance  of  this  representation  of  the  partition 
space,  as  it  is  called,  will  be  apparent  in  later  sections 
detailing  the  work  performed  in  this  program.  The  zero  and 
one  values  of  the  binary  input  variables  are  used  in  Table 
2.12  to  designate  the  variables  as  initial  row  or  column 
partition  selections,  respectively. 

Table  2.12.  Minimum  Sum  of  Cardinality  that  Can  Result  from 
All  Possible  Initial  Partition  Matrices  of  the  Original 

Function 


a 

0 

0 

0 

0 

1 

1 

1 

1 

b 

0 

0 

1 

1 

0 

0 

1 

1 

c 

0 

1 

0 

1 

0 

1 

0 

1 

def 

000 

64 

64 

64 

64 

64 

64 

64 

001 

64 

64 

64 

64 

64 

64 

64 

64 

010 

64 

64 

64 

64 

64 

64 

64 

64 

Oil 

64 

20 

64 

32 

64 

32 

64 

32 

100 

64 

64 

64 

64 

64 

64 

20 

32 

101 

64 

64 

64 

64 

64 

64 

32 

32 

110 

64 

64 

64 

64 

64 

64 

32 

32 

111 

64 

32 

64 

32 

64 

32 

32 
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There  does  not  appear  to  be  any  direct  method  of 
selecting  a  partition  that  will  decompose  to  the  DFC.  All 
the  partition  matrices  must  be  examined.  As  mentioned,  the 
nximber  of  possible  partition  matrices  for  a  given  binary 
function  is  2^  -  2  or  the  cardinality  minus  two.  This 
increases  exponentially  as  N,  the  number  of  input  variables, 
increases.  Each  of  these  partition  matrices  will  have  a 
number  of  elements  that  must  be  compared  in  calculating 
column  multiplicity.  This  also  increases  exponentially. 

This  rapidly  becomes  a  computationally  intense  problem  as 
the  number  of  input  variables  increases.  The  computational 
burden  increases  even  more,  since  each  new  component 
function  that  is  generated  must  also  be  decomposed. 

The  probability  of  finding  the  DFC  without  using  any 
information  about  the  function,  the  partition  space  or 
previously  tried  decompositions  is  no  better  than  random 
draw  or  "chance  without  replacement."  It*s  like  trying  to 
pick  a  black  ball  out  of  a  container  that  has  a  few  black 
balls  and  a  large  number  of  white  ones.  Initially  the 
probability  of  picking  a  white  ball  is  very  high.  As  more 
white  balls  are  selected  the  chance  of  getting  a  black  ball 
becomes  greater.  The  following  equation  illustrates  this. 

The  probability,  P,  of  finding  the  DFC  on  the  ith  try: 

P  =  Ndfc/(N  +  1  -  i) 


where : 

Ndfc  is  the  number  of  partition  matrices  which  lead  to 
the  DFC,  and 

N  is  the  total  nximber  of  partition  matrices. 

As  the  search  continues,  the  probability  of  finding  the 
DFC  increases.  The  probability,  P,  of  finding  the  DFC 

by  the  ith  try  is: 

N-Ndfc+1 

P  =  1  -nP[(N-  Ndfc  +  1  -  i)  /  (N  +  1  -  i)  ] 

i=l 

If  we  had  some  a  priori  knowledge  about  the 
distribution  of  partition  selections  that  led  to  the  DFC,  we 
could  make  more  intelligent  partition  selections.  If  we 
could  use  information  about  cardinality  as  we  made  partition 
selections,  we  could  improve  our  chances  of  finding  a  DFC 
more  quickly.  The  experiments  conducted  in  this  program, 
which  will  be  reported  in  later  sections,  compare  some 
techniques  for  utilizing  known  information  to  make  wiser 
partition  selections.  These  techniques  will  be  compared  to 
a  pure  "chance  without  replacement"  approach,  which  does  not 
try  to  base  partition  selection  on  any  previous  information. 


13 


2.2.1  Partition  Space 

The  array  shown  in  Table  2.12  is  slightly  rearranged  in 
Table  2.13  by  exchanging  the  labels  c  and  d.  Two 
dimensional  graphic  representations  of  Tables  2.12  and  2.13 
are  shown  in  Figures  2.8  and  2.9  respectively.  In  Figures 
2.8  and  2.9  a  higher  stack  of  blocks  represents  a  better 
decomposition.  There  appears  to  be  some  form  of  structure 
in  Figure  2.8.  The  structure  becomes  more  obvious  in  Figure 
2.9.  The  difference  between  Figures  2.8  and  2.9  is  only  a 
convenience  of  representation.  There  is  no  a  priori  way  of 
determining  which  selection  of  variables  for  axis  labels 
will  make  the  structure  more  apparent.  The  structure  that 
will  be  produced  by  any  function  is  unknown  until  the 
decompositions  are  performed. 

The  same  phenomenon  is  shown  for  a  slightly  more 
complex  function  in  Figures  2.10  and  2.11.  A  typical 
partition  space  for  an  eight  variable  function  is  shown 
graphically  in  Figure  2.10.  There  appears  to  be  some 
structure,  but  it  is  not  obvious.  Figure  2.11  shows  the 
same  function  with  the  labels  rearranged  to  make  the 
structure  more  obvious.  This  representation  of  the  function 
is  characterized  by  narrow  ridges  and  plateaus.  Gradient 
techniques  typically  used  for  finding  maxima  or  minima  do 
not  work  well  with  this  type  of  function  due  to  the  large 
number  of  discontinuities.  The  gradient  technique  is  most 
useful  when  there  are  gradual  changes  leading  to  maxima  and 
minima.  These  figures  show  a  two  dimensional  representation 
of  an  N  dimensional  space,  where  N  is  the  number  of  input 
variables. 

Table  2.13.  Rearrangement  of  Labels  c  and  d  in  Table  2 . 12 


a 

0 

0 

0 

0 

1 

1 

1 

1 

b 

0 

0 

1 

1 

0 

0 

1 

1 

d 

0 

1 

0 

1 

0 

1 

0 

1 

cef 

000 

64 

64 

64 

64 

64 

64 

20 

001 

64 

64 

64 

64 

64 

64 

64 

32 

010 

64 

64 

64 

64 

64 

64 

64 

32 

Oil 

64 

64 

64 

64 

64 

64 

64 

32 

100 

64 

64 

64 

64 

64 

64 

64 

32 

101 

64 

64 

64 

64 

64 

64 

64 

32 

110 

64 

64 

64 

64 

64 

64 

64 

32 

111 

20 

32 

32 

32 

32 

32 

32 
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Figure  2.8.  2D  Representation  of  Partition  Space  from  Table 

2.12. 


Figure  2.9.  2D  Representation  of  Partition  Space  from  Table 

2.13. 


Figure  2.10.  2D  Representation  of  a  Typical  Partition  Space 
of  an  Eight  Variable  Function. 


Figure  2.11.  2D  Representation  of  the  Same  Eight  Variable 
Function  With  Labels  rearranged  to  Make  the  Structure  in 
Partition  Space  More  Obvious. 


2.2.2  Search  Techniques 

A  niunber  of  techniques  for  searching  the  partition 
space  exist.  They  usually  involve  generating  a  partition, 
evaluating  it,  and  then  trying  another.  The  problem  with 
many  of  these  techniques  is  that  they  use  little  information 
about  the  function,  the  decompositions  that  were  previously 
found  or  structure  of  the  partition  space.  When  this 
information  about  the  function  is  not  used,  it  is  like 
trying  to  select  a  good  partition  by  pure  chance.  The 
following  is  a  description  of  various  techniques  for 
searching  the  partition  space,  some  which  use  information 
about  the  function  and  others  that  do  not.  Comments  are 
also  provided  about  their  efficacy. 

2. 2. 2.1  Estimation  of  DEC 

Estimate  the  DEC  based  on  column  multiplicity  or  some 
other  measure  of  the  likelihood  of  a  particular  partition 
leading  to  a  low  complexity.  Then  only  decompose  partition 
matrices  that  are  likely  to  decompose  well.  This  relieves 
some  of  the  computational  burden  and  can  be  used  in 
combination  with  some  of  the  following  techniques.  This 
technique  often  finds  the  DEC  or  near  optimal 
decompositions.  However,  it  does  not  always  find  the 
optimal  decomposition. 

2. 2. 2. 2  Random  (Chance) 

Randomly  generate  partition  selections.  This  is 
obviously  no  better  than  chance.  These  techniques  may  be 
viable  if  there  are  a  large  number  of  partition  matrices 
that  decompose  to  the  DEC.  A  variation  of  the  chance 
procedure  is  termed  "chance  without  replacement."  "Chance 
without  replacement"  is  equivalent  to  an  exhaustive  search, 
where  nothing  is  done  to  improve  the  search  procedure. 

2. 2. 2. 3  Decreasing  Row  to  Column  Ratio 

Select  partition  columns  first  using  only  one  variable 
for  a  column,  then  all  combinations  of  variables  two  at  a 
time,  then  three  at  a  time  and  so  on  up  to  the  number  of 
inputs  minus  one  at  a  time.  It  formed  the  basis  for  a 
version  of  the  Advanced  Eunction  Decomposition  algorithm 
used  in  phase  one  of  Pattern  Theory.  There  is  no  reason  to 
believe  that  this  is  any  better  than  chance.  In  fact  it 
appears  to  be  worse  than  chance.  This  technique  will  be 
documented  in  a  technical  report  titled  Pattern  Theory  II. 
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2. 2. 2. 4  Increasing  Row  to  Column  Ratio 

Select  partition  matrices  first  using  only  one  variable 
for  a  row,  then  all  combinations  two  at  a  time  for  rows, 
then  three  at  a  time  and  so  on  up  to  the  number  of  inputs 
minus  one  at  a  time.  There  is  empirical  evidence  that  this 
is  much  better  than  chance.  Selecting  partition  matrices  in 
an  increasing  row  to  column  ratio  may  capitalize  on  some 
underlying  structure  in  the  partition  spacet^]. 

2. 2. 2. 5  Correlation 

Perform  a  correlation  of  the  values  of  input  variables 
and  values  of  the  function.  Then  select  groups  of  input 
variables  with  the  same  correlation  coefficients  for  either 
rows  or  columns.  This  technique  differs  from  those 
previously  mentioned.  It  often  significantly  limits  the 
search  space  based  on  information  about  the  function.  There 
is  evidence  that  this  technique  usually  leads  to  low 
complexity  decompositions  and  is  much  better  than  chance  for 
finding  the  DFC.  However,  there  is  no  guarantee  that  any  of 
the  partition  matrices  selected  by  this  method  will  find  the 
optimal  DFC  and  in  some  cases  this  method  does  not  provide 
any  information  to  guide  the  search.  A  technical  paper 
describing  this  technique  is  provided  in  Appendix  A. 

2 . 2 . 2 . 6  Genetic 

First,  generate  some  decompositions  by  one  of  the 
previous  methods.  Then  partially  exchange  some  of  the  row 
or  column  variables  in  partition  matrices  that  previously 
decomposed  to  low  complexity.  This  should  generate  new 
partition  matrices  that  are  more  likely  to  decompose  to  low 
complexity.  This  is  the  basic  principle  behind  the 
application  of  genetic  algorithms  to  partition  selection. 

It  essentially  uses  information  about  previously  found 
decompositions.  Genetic  algorithms  appear  to  be  a  promising 
technique  for  partition  selection  and  were  the  focus  of 
experiments  conducted  in  this  program.  The  next  section 
gives  some  background  information  on  genetic  algorithms. 

2.3  Genetic  Algorithms 

This  section  describes  genetic  algorithms.  It  gives 
the  general  flow  of  the  algorithm.  It  also  provides  the 
definition,  description  and  effects  of  some  of  the 
parameters  and  heuristics  used  in  genetic  algorithms. 

2.3.1  Comparisons  with  Biological  Evolution 

Genetic  algorithms  are  an  attempt  to  find  solutions  to 
particularly  difficult  problems  by  using  modeling  mechanisms 
patterned  after  biological  evolution.  A  collection  or  pool 
of  candidate  solutions  to  a  given  problem  is  referred  to  as 
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the  population.  Each  member  of  the  population,  representing 
a  particular  solution  to  the  problem,  is  called  a 
chromosome.  A  representation  of  the  problem  space  is 
selected  that  has  interchangeable  units  or  features  that 
characterize  the  solution.  The  interchangeable  units  are 
equivalent  to  genes  in  the  biological  model.  The  specific 
value  that  a  gene  takes  on  is  called  an  allele  and  often 
represents  some  characteristic  of  the  solution. 

Biologically,  the  chromosomes  are  string-like  bodies 
containing  the  genes  of  an  individual.  Each  gene  refers  to 
some  characteristic  of  the  individual  such  as  eye  color. 

The  allele  is  the  specific  representation  of  the 
characteristic,  such  as  blue  or  brown  for  the  gene 
designating  eye  color.  In  genetic  algorithms  each  solution 
or  chromosome  is  made  up  of  a  string  of  bits,  integers,  or 
characters.  The  individual  bits,  integers,  or  characters 
are  the  genes  and  the  specific  value  of  each  gene  is  the 
allele. 

An  initial  population  is  generated  by  random  or 
heuristic  methods.  The  population  is  then  evaluated  to  find 
the  chromosomes  that  yield  good  solutions.  These 
chromosomes  are  selected  for  survival  and  reproduction, 
usually  by  random  draw.  Chromosomes  yielding  less  fit 
solutions,  tend  to  die  off.  This  is  similar  to  survival  of 
the  fittest  in  Darwinian  evolution.  Alleles  can  be 
exchanged  between  selected  chromosome  pairs,  as  in 
biological  crossover,  or  genes  may  be  modified,  as  in 
mutation.  Thus,  a  new  population  is  generated.  Each  new 
population  is  called  a  generation.  Each  generation  will 
contain  some  of  the  chromosomes  that  were  good  solutions 
from  the  old  population  and  new  chromosomes  that  should  have 
a  high  probability  of  being  better  solutions  to  the  problem. 
This  process  is  repeated  for  each  new  generation  until  some 
terminating  condition  is  reached.  In  each  new  generation 
the  number  of  better  solutions  increases  and  should 
predominate  the  population.  After  a  number  of  generations, 
the  population  should  converge  to  the  best  solutions.  A 
high-level  flow  diagram  of  the  genetic  algorithm  process  is 
provided  in  Figure  2.12. [ 2 ] 
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Figure  2.12.  Basic  Genetic  Algorithm  Process. 


2.3.2  Components  of  Genetic  Algorithms 

There  are  three  major  components  to  the  application  of 
genetic  algorithms:  representation,  evaluation,  and  the 
mechanics  of  reproduction.  "In  order  to  employ  a  genetic 
algorithm  the  analyst  must  supply  three  items:  1.  The 
representation  of  the  problem  as  a  string  of  digits,  2.  A 
means  of  evaluating  individual  solutions,  3.  The  specifics 
of  the  crossover  operator.  •' [^1 
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2. 3. 2.1  Representation  of  the  Problem 

The  representation  of  individual  chromosomes  can  be 
strings  of  binary  digits,  lists  of  integers,  real  numbers, 
rules  or  other  symbols.  Binary  digits  are  considered  to  be 
the  most  general  representation  and  evidence  exists  that  the 
binary  representation  is  optimal. [4/5]  Representation  of  a 
problem  or  solution  is  perhaps  the  most  formidable  aspect  of 
designing  a  genetic  algorithm.  It  is  one  of  the  areas  that 
require  expertise  in  the  problem  domain  and  can  seriously 
affect  the  performance  of  the  algorithm.  Important  features 
of  the  problem  must  be  represented  in  such  a  way  that 
desirable  solution  characteristics  are  propagated,  and 
undesirable  characteristics  suppressed. 13]  The 
representation  for  chromosomes  in  the  program  discussed  in 
this  report  uses  binary  strings  with  the  same  number  of 
digits  as  the  number  of  input  variables  for  a  function.  As 
Section  3.0  will  explain,  the  value  of  each  individual 
allele  represents  an  initial  row  or  column  partition 
selection  for  the  corresponding  binary  input  variable. 

2. 3. 2. 2  Evaluating  Individual  Solutions 

Evaluation  of  individual  solutions  or  determining  the 
fitness  of  the  chromosomes  can  be  performed  in  a  number  of 
ways.  If  the  algorithm  is  to  perform  some  game  or  task, 
tournaments  can  be  conducted  with  the  winners  or  good 
performers  selected.  This  is  similar  to  competition  and 
survival  of  the  fittest  in  biologic  evolution.  If  the 
algorithm  is  attempting  to  compute  some  function  or  solve  an 
optimization  problem  that  can  be  computed  in  some  other  way, 
then  the  chromosomes  that  are  closest  to  the  actual  result 
are  selected.  This  is  like  exploitation  of  an  environmental 
niche  in  biologic  evolution.  Of  course,  all  this  implies 
that  some  independent  measure  of  performance  exists. 

2 . 3 . 2 . 3  Crossover  Operator 

Before  examining  some  details  of  the  evaluation  and 
selection  process,  such  as  the  number  of  survivors  or  how 
many  new  members  are  generated,  an  examination  of 
reproduction  mechanisms  will  be  performed.  This  examination 
focuses  specifically  on  the  crossover  operator.  Crossover 
is  a  procedure,  whereby  chromosome  or  binary  string  pairs 
exchange  alleles  to  produce  new  chromosomes  that  may  provide 
better  solutions  to  the  problem.  Crossover  techniques  for  a 
binary  string  are  fairly  standard.  Single  and  double 
crossover  are  most  commonly  used  in  practice.  These 
techniques  are  illustrated  in  Tables  2.14  and  2.15. 

In  single  crossover  (Table  2.14)  a  crossover  point  is 
randomly  selected  at  the  same  position  in  two  binary 
functions  or  chromosomes,  referred  to  as  Parents  A  and  B. 
Child  AB  is  produced  by  selecting  the  alleles  in  front  of 
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the  crossover  point  from  Parent  A  and  the  alleles  following 
the  crossover  point  from  parent  B.  Often  a  reciprocal 
alleleomorph  Child  BA  is  produced  by  selecting  the  alleles 
in  front  of  the  crossover  point  from  Parent  B  and  the 
alleles  following  the  crossover  point  from  Parent  A. 

Table  2.14.  Single  Crossover. 


Parent 

A 

01001010 

010101001010010100 

Parent 

B 

10010101 

001010010010010100 

Child 

AB 

01001010 

001010010010010100 

Child 

BA 

10010101 

010101001010010100 

In  double  crossover  (Table  2.15)  two  crossover  points 
are  randomly  selected.  Child  ABA  is  produced  by  selecting 
the  alleles  in  front  of  the  first  crossover  point  and  in 
back  of  the  last  crossover  point  from  Parent  A.  The  alleles 
between  the  first  and  second  crossover  points  are  selected 
from  Parent  B.  Again  a  reciprocal  alleleomorph.  Child  BAB, 
can  be  produced  by  selecting  the  alleles  in  front  of  the 
first  crossover  point  and  in  back  of  the  last  crossover 
point  from  Parent  B.  The  alleles  between  the  first  and 
second  crossover  points  are  selected  from  Parent  A. 

Table  2.15.  Double  Crossover. 


Parent  A 

01001 

0100101 

01001010010100 

Parent  B 

10010 

1010010 

10010010010100 

Child  ABA 

01001 

1010010 

01001010010100 

Child  BAB 

10010 

0100101 

10010010010100 

2. 3. 2. 4  Mutation  Operator 

Typically  in  genetic  algorithms  each  fundamental  unit 
or  gene  has  a  finite  probability  of  changing.  This 
increases  the  variability  of  a  population.  In  binary 
representations  mutation  usually  consists  of  changing  a  bit 
or  bits.  Mutation  can  avoid  solutions  that  lock  on  local 
minima  and  result  in  a  search  of  the  entire  problem  space. 

2 . 3 . 2 . 5  Other  Parameters  and  Heuristics 

Once  the  criteria  for  the  representation,  evaluation, 
and  crossover  operator  have  been  defined,  there  are  still  a 
number  of  parameters  and  heuristics  that  influence  the 
algorithm's  performance C ^ .  These  parameters  often  interact 
with  each  other,  and  the  design  of  a  genetic  algorithm 
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becomes  an  iterative  process.  A  description  of  these 
parameters  is  given  below. 

Population  size:  The  population  size  chosen  affects 
the  global  performance  and  efficiency  of  a  genetic 
algorithm.  A  large  population  is  desirable  to  insure 
sufficient  coverage  of  the  problem  space.  A  small 
population  is  likely  to  prematurely  converge  on  a  local 
rather  than  global  solution. 

Crossover  rate:  This  is  defined  as  the  percentage  of 
the  population  that  may  be  modified  by  the  crossover 
operator.  If  the  crossover  rate  is  too  high,  good 
performers  are  removed  faster  than  selection  can  produce 
improvements.  If  the  rate  is  too  low,  the  search  may 
stagnate . 

Mutation  rate:  The  rate  or  probability  of  a  mutation 
occurring.  A  low  level  of  mutation  prevents  a  gene  from 
becoming  locked  into  a  particular  allele  and  helps  to  insure 
that  the  entire  problem  space  is  searched.  A  mutation  rate 
that  is  too  high  results  in  a  random  search. 

Generation  gap:  The  generation  gap  is  the  percentage 
of  the  population  that  is  replaced  each  generation.  A 
generation  gap  of  one  (100%)  means  that  the  entire 
population  would  be  replaced.  A  generation  gap  that  is  too 
high  could  result  in  good  performers  being  removed.  The 
other  extreme  is  a  generation  gap  of  zero.  This  would  mean 
no  replacement  and  is  quite  useless,  since  there  would  be  no 
possibility  of  evolution  toward  a  solution. 

Fitness  factor:  This  is  a  measure  of  performance  of 
the  individual  chromosomes.  It  is  often  used  to  control 
both  survival  and  reproduction.  Survival  and  reproduction 
can  also  have  separate  fitness  factors. 

Selection  strategy:  The  logic  or  heuristics  that 
select  individuals  for  survival  and  reproduction.  Two 
common  strategies  are  select  for  survival  first  or  select 
for  reproduction  first.  In  select  for  reproduction  first, 
all  the  chromosomes  have  a  chance  of  reproducing.  In  select 
for  survival  first  only  those  which  survive  reproduce. 

Elitist  strategy:  A  survival  strategy  which  insures 
that  the  best  performers  survive. 

Scaling  factor:  A  scaling  factor  can  be  used  to  adjust 
the  fitness  factor  to  prevent  early  dominance  by  particular 
individuals,  so  the  problem  space  is  sufficiently  explored. 
The  scaling  factor  normalizes  or  provides  a  measure  of 
relative  fitness.  If  the  scaling  factor  is  too  low  the 
search  may  be  near  random  and  very  slow.  If  this  factor  is 
too  high  convergence  may  occur  at  a  local  minimum. 
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Population  diversity  during  evolution  is  important  and  a 
scaling  factor  helps  maintain  this  diversity. 

Ranking:  An  alternative  to  Fitness  and  scaling  which 
ranks  or  sorts  members  of  the  population  based  on 
performance. 

Evolutionary  programming:  A  type  of  genetic  algorithm 
which  uses  high  mutation  rates,  often  excludes  crossover 
completely,  a  small  population,  elitist  strategy,  and  a  very 
low  generation  gap. 
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3.0  Application  of  Genetic  Algorithms  to  Partition  Selection 


This  section  describes  how  genetic  algorithms  were 
applied  to  partition  selection  in  function  decomposition. 
Genetic  algorithms  seem  to  be  a  real  natural  for  partition 
selection.  They  provide  a  powerful  tool  for  finding  near 
optimal  answers  in  combinatorial-type  problems  without 
exhaustively  searching  through  potentially  large  solution 
spaces.  This  type  of  optimization  is  precisely  the  problem 
encountered  in  partition  selection  as  a  search  for  the 
simplest  decomposition  is  conducted.  Specific  examples  why 
genetic  algorithms  provide  a  useful  mechanism  for  partition 
selection  includes  the  ease  with  which  row  and  column 
selection  can  be  represented  by  a  binary  string.  Also,  the 
cardinality  in  function  decomposition  provides  a  natural 
fitness  factor  that  must  be  defined  when  developing  a 
genetic  algorithm.  Additionally,  genetic  algorithms  are  not 
gradient  techniques,  which  as  mentioned  earlier,  do  not  work 
well  with  the  search  space  in  this  type  of  problem. 

In  applying  genetic  algorithms  to  partition  selection 
it  was  necessary,  as  mentioned  in  Section  2.3.2,  to  properly 
address  the  representation  of  the  problem,  the  evaluation  of 
individual  solutions,  and  the  mechanics  of  reproduction  such 
as  the  crossover  operator.  The  following  describes  the  two 
primary  approaches  taken  to  define  these  characteristics  as 
well  as  other  genetic  algorithm  parameters  and  heuristics 
examined  in  this  program.  Both  of  the  algorithms  chosen 
used  some  evolutionary  programming  techniques.  The  approach 
referred  to  as  the  structured  genetic  algorithm  was  highly 
evolutionary.  This  algorithm  varied  more  from  methods 
classically  used  in  applying  genetic  algorithms.  The  other 
genetic  algorithm  was  referred  to  as  the  baseline  genetic 
algorithm.  The  differences  between  the  two  approaches  will 
be  discussed  with  special  attention  given  to  the  variations 
from  classical  methods.  Several  design  decisions  were  made, 
because  of  the  nature  of  the  application,  that  may  have 
limited  the  full  evaluation  of  genetic  algorithms  in  some 
cases.  In  other  words,  a  complete  parametric  analysis  was 
not  performed.  Section  4.0  will  later  provide  a  comparison 
of  the  results  from  both  the  baseline  and  structured  genetic 
algorithm  approaches. 

3.1  Representation  of  the  Problem 

As  previously  mentioned,  binary  strings  are  favored  in 
theory  as  well  as  in  practice  to  represent  chromosomes.  In 
a  like  manner,  for  the  partition  selection  problem  in 
function  decomposition,  chromosomes  can  be  represented  by 
binary  strings.  Recalling  the  example  in  Section  2.2.1, 
Table  3.1  shows  the  minimum  sum  of  cardinality  that  can 
result  from  all  possible  initial  partition  matrices  of  the 
original  function  represented.  The  table  shown  is  a 


24 


representation  of  the  partition  space  for  a  function  with 
six  binary  input  variables.  In  the  genetic  algorithms 
developed  in  this  program  chromosomes  were  used  to  specify 
locations  in  such  a  partition  space.  Each  location  in  this 
space  correspondingly  refers  to  a  specific  initial  partition 
selection.  To  specify  a  chromosome,  each  input  variable 
(a,b,c,d,e  and  f  in  this  example)  is  assigned  a  position  in 
a  binary  string.  The  binary  string  has  as  many  bits  as 
there  are  input  variables  (six  in  this  case) .  In  the  binary 
representation  a  "one”  means  the  corresponding  input  is 
selected  as  a  column  variable  and  a  "zero"  means  the  input 
is  selected  as  a  row  variable.  This  provides  a  very  natural 
and  compact  way  of  defining  the  partition  matrices  to  be 
evaluated.  In  the  genetic  algorithms  when  a  crossover  or 
mutation  is  performed  the  new  chromosome  represents  a  new 
location  in  the  partition  space.  Likewise,  the  chromosome 
also  represents  another  initial  partition  selection  that 
attempts  to  decompose  the  original  function  to  a  lower 
cardinality. 

Table  3.1.  Two  Dimensional  Table  Representation  of  the 
Partition  Space  for  a  Function  With  Six  Binary  Input 

Variables . 


a 

0 

0 

0 

0 

1 

1 

1 

1 

b 

0 

0 

1 

1 

0 

0 

1 

1 

d 

0 

1 

0 

1 

0 

1 

0 

1 

cef 

000 

64 

64 

64 

64 

64 

64 

20 

001 

64 

64 

64 

64 

64 

64 

64 

32 

010 

64 

64 

64 

64 

64 

64 

64 

32 

oil 

64 

64 

64 

64 

64 

64 

64 

32 

100 

64 

64 

64 

64 

64 

64 

64 

32 

101 

64 

64 

64 

64 

64 

64 

64 

32 

110 

64 

64 

64 

64 

64 

64 

64 

32 

111 

20 

32 

32 

32 

32 

32 

32 

In  the  current  example,  the  locations  in  the  partition 
space  and  corresponding  initial  partition  selections  that 
yield  the  DFC  (lowest  sum  of  cardinality)  of  20  are 
represented  by  the  following  chromosomes: 

Binary  a  b  c  d  e  f 

Variables 

Chromosome  1  110100 

Chromosome  2  001011 

In  one  optimal  initial  partition  selection  a,  b  and  d  should 
be  selected  as  column  variables  and  c,  e,  and  f  should  be 
row  variables.  In  the  other  optimal  partition  selection  a. 
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b  and  d  should  be  row  variables  and  c,  e  and  f  should  be 
selected  as  colxomn  variables.  A  decomposition  starting  with 
these  initial  partition  selections  will  lead  to  subfunctions 
having  the  lowest  sum  of  cardinality  possible. 

3.2  Evaluating  Individual  Solutions 

Evaluation  of  individual  solutions,  or  fitness  of  the 
chromosomes,  is  also  quite  natural.  It  can  be  done  by  simply 
performing  the  function  decomposition  of  the  partition 
specified  by  the  chromosome  and  obtaining  the  sum  of 
cardinality.  The  chromosome  yielding  a  partition  with  the 
lowest  cardinality  is  the  most  fit.  Column  multiplicity  or 
partially  decomposing  a  function  can  also  be  used  as 
evaluation  factors  to  save  the  computation  of  doing  the 
multi  level  decomposition.  However,  if  these  factors  are 
used,  the  result  is  an  estimate  rather  than  actual 
cardinality.  A  simulated  DEC  and  an  estimate  of  DEC  based 
on  one  level  of  decomposition  were  used  as  evaluation 
factors  in  this  effort. 

3.3  Crossover  Operator 

This  leaves  the  specifics  of  the  crossover  operator. 
Single  and  doxible  crossovers  were  tried  in  this  program. 
Uniform  crossover  was  also  tried  in  the  baseline  genetic 
algorithm.  The  following  example  (Table  3.2)  illustrates 
uniform  crossover  in  a  chromosome  corresponding  to  a 
function  with  26  input  variables.  In  uniform  crossover 
Child  AB  is  produced  by  randomly  selecting  alleles  from 
Parent  A  or  B.  A  reciprocal  alleleomorph  was  produced  by 
selecting  alleles  from  the  opposite  parent. 

Table  3.2.  Uniform  Crossover. 


01001010010101001010010100 

10010101001010010010010100 


11000010000010010010010100 

babaaaaaabaabbbbbabaababbb 


00011101011101001010010100 

ababbbbbbabbaaaaababbabaaa 


Parent  A 
Parent  B 

Child  AB 
alleles 

Child  BA 
alleles 
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A  structured  crossover  operator  was  tried  (Table  3.3) 
in  the  structured  genetic  algorithm.  This  technique  is 
named  as  such,  because  it  provides  a  more  structured  search. 
In  this  form  of  crossover  the  different  alleles  are  counted. 
The  crossover  point  is  not  random,  but  each  child  receives 
approximately  one  half  the  different  alleles  from  each 
parent.  In  each  generation  the  number  of  differences  is 
reduced  which  should  speed  up  convergence  of  the  algorithm. 


Table  3.3.  Structured  Crossover . 


Parent  A 
Parent  B 
differences 
crossover 


000  01000 
000  10101 
XXX  X 


01000101000  0010100 
00001001001  0010100 
X  XX  X 


Child  AB  000  01000 
Child  BA  000  10101 


00001001001  0010100 
01000101000  0010100 


second  generation 


Parent  A 

000  01 

000  010001010000010100 

Child  BA 

000  10 

101  010001010000010100 

XX 

X  X 

Child  ABA 

000  01 

101  010001010000010100 

Child  BAA 

000  10 

000  010001010000010100 

The  genetic  algorithms  evaluated  in  this  program  were 
intended  to  apply  to  large  numbers  of  input  variables. 
However,  to  prevent  population  sizes  from  getting  very 
large,  the  duplication  of  chromosomes  was  not  allowed. 
Instead,  when  a  mating  produced  a  duplicate  chromosome 
several  options  were  exercised; 

a)  Baseline  Genetic  Algorithm 

o  the  mating  was  tried  several  more  times  with  the 
same  parents  or 

o  a  mutation  was  produced 

b)  Structured  Genetic  Algorithm 

o  the  duplicate  chromosome  was  ignored  and  another 
mating  was  tried  with  different  parents,  otherwise 

the  structured  process  would  produce  exactly  the 

same  children 
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3.4  Heuristics  and  Parameter  Selection 

Many  other  heuristics  had  to  be  established  and 
parameters  set  to  apply  the  genetic  algorithm  methodology  to 
partition  selection.  Among  these  parameters  and  heuristic 
strategies  were  those  defined  in  Section  2. 3. 2. 4  and 
2. 3. 2. 5.  The  following  discussion  specifies  how  these 
parameters  and  strategies  were  applied  to  the  partition 
selection  problem.  This  includes  the  establishment  of  the 
initial  chromosome  population,  selection  strategies  and 
rates  for  reproduction,  and  the  method  used  to  judge  the 
fitness  of  the  individual  chromosomes.  Both  the  baseline 
and  structured  genetic  algorithm  approaches  are  addressed. 

A  summary  of  the  differences  between  the  two  approaches  is 
given  in  Table  3.4.  A  good  find  refers  to  a  chromosome 
representing  a  partition  that  yields  a  new  cardinality  lower 
than  the  original.  The  following  discussion  will  further 
clarify  these  comments. 
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Table  3.4.  Six  Key  Differences  Between  the  Baseline  and 
Structured  Genetic  Algorithm  Approaches  to  Partition 

Selection. 


BASELINE 

STRUCTURED 

Initial  population  is 
generated  by  taking  first 
good  find,  compliment  of 

1st  half  and  2nd  half  of 
first  good  find. 

Po  =  3  +  # random  bad  finds 

Po:  #  Chromosomes  in 
initial  population. 

Initial  population  is 
generated  by  taking  first 
good  find  and  individual 
compliment  of  each  of  its 
bits. 

Po  =  #bits  +  1  +  #random 
bad  finds 

Po:  #  Chromosomes  in 
initial  population. 

Each  member  of  population 
has  some  (tailored) 
probability  of  reproducing 
based  on  a  fitness  ranking. 

Only  the  best  reproduce  in 
an  ordered  fashion. 

Population  is  never 
reinitialized . 

Population  is  reinitialized 
whenever  a  better 
chromosome  is  found. 

Reproduction  is  via  uniform 
crossover. 

Reproduction  is  via 
structured  single 
crossover . 

Mutations  occur  if  after  N 
tries  (usually  a  small 
number  like  2  or  3)  parents 
do  not  produce  a  new 
chromosome . 

Mutation  never  occurs. 

No  (real)  terminating 
condition.  Search  stops  if 
solution  is  found  or  after 

2^  generations 

Terminates  if  all  matings 
of  "best"  occur  and  nothing 
better  is  found.  Algorithm 
may  terminate  before 
finding  DEC. 
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3.4.1  Population  Size  and  Setup 

To  fora  the  initial  population  in  both  the  baseline  and 
structured  genetic  algorithms,  random  chromosomes  were 
generated  until  one  was  found,  which  represented  a 
cardinality  lower  than  the  initial  cardinality.  All  the 
chromosomes  were  stored  in  a  population  array.  A  population 
was  then  generated  from  the  chromosome  representing  the 
lowest  cardinality.  If  the  array  became  full  the  "least 
fit"  (highest  cardinality)  chromosomes  were  removed.  No 
upper  limit  was  placed  on  population  size  in  the  genetic 
algorithms  actually  evaluated.  Also  no  tradeoff  studies 
were  performed  on  how  population  size  affects  convergence  of 
the  algorithm.  Several  methods  for  generating  the  initial 
population  and  subsequent  populations  were  tried: 

1)  The  compliment  of  an  existing  chromosome  was  taken  to 
generate  another  member  of  the  population. 

00011101  Parent  I 

11100010  Compliment  Parent  II 

2)  In  the  baseline  genetic  algorithm  the  compliment  of  each 
half  of  the  parent  was  taken. 

00011101  Parent  A 
11101101  Child  A 

00010010  Child  B 

It  was  hoped  that  by  doing  this  a  few  good  choices  would  be 
found  early.  Several  other  variations  of  this  method  were 
tried.  One  variation  included  dividing  the  Parent 
chromosome  into  thirds  and  forming  compliments  of  the 
various  sections. 


00 

0111 

01 

Parent 

A 

11 

0111 

10 

Child 

C 

00 

1000 

01 

Child 

D 

3)  In  the  stiructural  search  algorithm  the  compliment  of  each 
of  the  input  variables  was  taken,  one  at  a  time  to  generate 
the  initial  population: 

00011101  Parent 
10011101  Child  A 
01011101  Child  B 
00111101  Child  C 
00001101  Child  D 
00010101  Child  E 
00011001  Child  F 
00011111  Child  G 
00011100  Child  H 
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The  purpose  of  these  methods  was  to  set  up  an  initial 
population,  which  can  explore  the  entire  space.  Since  the 
initial  solution  is  complimented,  some  crossover  can  provide 
any  possible  combination  of  row/ column  choices.  Variation 
1)  provides  the  smallest  initial  population,  but  also  has 
the  largest  initial  difference  between  chromosomes. 

Variation  3)  has  the  largest  initial  population  with  the 
smallest  difference  between  chromosomes. 

As  each  new  chromosome  was  generated,  a  comparison  was 
made  with  existing  chromosomes  in  the  population.  If  it  was 
not  already  in  the  population  the  new  chromosome  was  placed 
at  the  beginning  of  the  population  array.  The  population 
was  then  either  fully  or  partially  sorted  (ranked)  by  one  of 
the  methods  described  below. 

The  following  description  of  the  sort  procedures  uses 
letters  to  represent  entire  chromosomes.  The  numbers  are 
fitness  values  that  are  stored  with  each  chromosome  in  the 
genetic  algorithms.  The  lower  the  fitness  value,  the  "more 
fit"  the  chromosome. 


As  an  example, 
value  of  3 

value  of  5 


Chromosome  A  may  be: 
Chromosome  B  may  be: 


010101  with  a  fitness 
101101  with  a  fitness 


1)  Full  Sort: 


A  full  sort  was  used  in  which  the  entire  population  was 
sorted  before  reproduction  occurred  again.  The  following 
example  illustrates  the  full  sort  for  a  population  of  8 
chromosomes . 

Before  Sort  A  B  C  D  E  F 

3  5  7  6  2  8 


After  Sort 


E  A  B  D  C  F 

2  3  5  6  7  8 


This  procedure  was  used  in  the  structured  genetic 
algorithm.  When  a  new  chromosome  was  added,  a  chromosome  at 
the  end  of  a  full  population  array  was  dropped  out  of  the 
array  as  shown  below. 


Before  sort 

Chromosome  G  produced: 

Add  new  chromosome  "4" 

Drop  last  chromosome  "8" 


A  B  C  D  E  F 

3  5  7  6  2  8 

G  A  B  C  D  E 

4  3  5  7  6  2 


After  Sort 


E  A  G  B  D  C 

2  3  4  5  6  7 
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2)  Partial  Sort  #1  (Compare  each  in  sequence) 


A  partial  sort  procedure  was  used  that  begins  with  the 
first  chromosome  in  the  population.  This  chromosome  is 
compared  with  the  one  directly  following  it  in  the 
population.  If  it  is  less  fit  it  moves  backward  in  the 
population.  It  is  then  compared  with  the  next  chromosome 
and  the  same  procedure  is  followed.  Whenever  it  is  more  fit 
than  the  chromosome  directly  following  it,  the  more  fit 
chromosome  retains  its  position.  This  represents  one 
partial  sort  cycle.  Reproduction  occurs  between  partial 
sort  cycles. 

Before  Sort  A  B  C  D  E  F 

3  5  7  6  2  8 


After  Partial 
Sort 


A  B  C  E  F  D 

3  5  6  2  7  8 


This  sort  procedure  was  tried  in  the  baseline  genetic 
algorithm.  After  a  number  of  cycles  this  partial  sort 
results  in  a  full  sort.  In  this  sort  poor  performers  are 
moved  right  to  the  end  of  the  population  and  do  not 
reproduce.  As  in  the  full  sort,  a  new  chromosome  at  the 
beginning  of  a  full  population  array  results  in  the  last 
chromosome  being  dropped. 


Before  sort 

A 

B 

C 

D 

E 

F 

3 

5 

7 

6 

2 

8 

Chromosome 

G  produced: 

Add  new  chromosome  "4" 

G 

A 

B 

C 

D 

E 

Drop  last  chromosome  "8” 

4 

3 

5 

7 

6 

2 

After  1st  Partial  Sort 

A 

G 

B 

C 

D 

E 

3 

4 

5 

7 

6 

2 

Chromosome 

H  produced: 

Add  new  chromosome  ”2'' 

H 

A 

G 

B 

C 

D 

Drop  last  chromosome  ”2" 

2 

3 

4 

5 

7 

6 

After  2nd  Partial  Sort 

H 

A 

G 

B 

D 

C 

2 

3 

4 

5 

6 

7 

3)  Partial  Sort  #2  (Pairwise  Flip) 

This  partial  sort  compares  chromosomes  within  a  pair 
for  fitness,  beginning  with  the  first  two  chromosomes  in  the 
population.  If  the  second  chromosome  is  more  fit  it  is 
flipped  to  the  first  position  in  the  pair.  After  one 
pairwise  sort  through  the  entire  population,  reproduction 
occurs . 
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Before  Sort 


A  B  C  D  E  F 

3  5  7  6  2  8 


After  Partial 
Sort 


A  B  D  C  E  F 

3  5  6  7  2  8 


This  sort  keeps  chromosomes  from  being  moved  to  the  end 
of  the  population  too  rapidly,  since  reproduction  is  based 
on  position  in  population  and  occurs  between  sort 
operations.  This  procedure  results  in  a  slower  sort  than 
the  others  and  can  even  get  stuck  if  nothing  were  added  to 
the  population.  However,  new  elements  are  placed  at  the 
beginning  of  the  population  after  each  partial  sort.  As  in 
the  other  sort  procedures,  the  last  element  is  dropped  from 
the  population  when  reproduction  occurs  if  the  population 
array  is  full. 


Before  sort 

Chromosome  G  produced: 

Add  new  chromosome  "4” 
Drop  last  chromosome  •'8" 


A  B  C  D  E  F 

3  5  7  6  2  8 

G  A  B  C  D  E 

4  3  5  7  6  2 


After  1st  Partial  Sort 

Chromosome  H  produced: 

Add  new  chromosome  "2” 
Drop  last  chromosome  ”6" 


A  G  B  C  E  D 

3  4  5  7  2  6 

H  A  G  B  C  E 

2  3  4  5  7  2 


After  2nd  Partial  Sort 


H  A  G  B  E  C 

2  3  4  5  2  7 


3.4.2  Crossover  Rate 

Only  two  members  of  the  population  were  mated  each 
generation.  Depending  on  the  crossover  scheme  implemented 
two  or  four  children  were  produced.  The  crossover  rate  was 
not  varied  from  one  generation  to  another - 

3.4.3  Mutation  Rate 


No  mutations  were  performed  in  the  structured  search 
algorithm  during  reproduction.  However,  if  a  more  fit 
chromosome  was  generated  the  population  was  reinitialized. 
This  procedure  effectively  conducts  a  mini  search  around  the 
most  fit  chromosome. [^3  In  the  baseline  algorithm  mutations 
were  performed,  but  not  by  the  use  of  a  set  mutation  rate. 
Rather  by  the  following  rule:  if  several  attempted  matings 
of  a  set  of  parents  failed  to  produce  a  unique  chromosome 
not  in  the  population,  a  mutation  occurred.  This  mutation 
consisted  of  changes  to  some  of  the  common  bits  between 
parents . 
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3.4.4  Generation  Gap 

Since  few  children  were  produced  the  generation  gap  was 
very  low.  The  generation  gap  was  kept  low  to  retain  good 
solutions  in  the  event  the  algorithm  does  not  converge  to 
optimal.  Near  optimal  solutions  are  desired. 

3.4.5  Fitness  Factor 

The  fitness  factor  for  both  genetic  algorithms  was 
defined  as  the  decomposed  cardinality  or  DFC.  Obviously, 
the  lower  the  cardinality  the  more  "fit”  the  chromosome  or 
the  better  the  decomposition. 

3.4.6  Selection  Strategy 

The  selection  strategy  was  based  on  the  order  of  the 
population  array  and  always  employed  select  for  reproduction 
before  survival.  As  mentioned,  the  population  array  was 
sorted  so  the  most  "fit"  chromosomes  were  at  the  front. 
Conversely  the  last  member  of  the  array  was  the  least  "fit" 
after  a  sort.  Also  the  last  member  of  an  array  was  removed 
each  time  a  child  was  produced  if  the  population  array  was 
full.  The  selection  strategy  depicted  in  Figure  3.1  was 
such  that  the  probability  of  selecting  a  parent  for 
reproduction  was  biased  toward  the  start  of  the  array.  This 
figure  shows  a  distribution  for  a  population  of  8 
chromosomes.  This  selection  strategy  was  referred  to  as  a 
"tailored  probability."  In  general  the  chance  of  being 
selected  for  reproduction  improves  with  fitness,  but  not 
strictly.  In  Figure  3.1  the  lower  the  Fitness  Factor  or  DFC 
the  "more  fit"  the  chromosome.  This  figure  represents  a 
population  with  hypothetical  Fitness  Factors  2,  4,  3,  5,  4, 

6 ,  6 ,  6 . 
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Figure  3.1.  Chance  of  Chromosome  Being  Selected  Compared  to 
Fitness  Factor  for  a  Population  of  8. 

In  the  structured  search  each  member  of  the  population 
was  mated  in  an  ordered  fashion.  Chromosomes  representing 
the  lowest  decompositions  found  so  far  were  mated  with  other 
"most  fit"  chromosomes.  Each  time  a  "better  fit"  was  found, 
the  population  was  reinitialized.  Several  excursions  were 
tried  in  the  structured  algorithm  which  allowed  mating  of 
slightly  "less  fit"  members  of  the  population.  The  details 
and  effects  of  these  excursions  will  be  described  in  a  later 
section. 

In  the  baseline  genetic  algorithm  several  candidates 
were  selected  for  reproduction  and  those  that  had  the  fewest 
different  (but  at  least  two)  allele  were  mated.  If  mating 
had  been  allowed  when  there  was  only  one  difference,  the 
children  would  be  a  duplicate  of  one  of  the  parents. 

3.4.7  Scaling  Factor 

The  Fitness  Factor  was  not  adjusted,  but  as  mentioned, 
the  most  fit  were  moved  to  the  beginning  of  the  population 
array. 
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4.0  Experiment  Description 


In  a  true  function  decomposition  a  function  is  broken 
all  the  way  down  to  find  the  DEC.  Since  the  time  required 
to  perform  full  function  decompositions  was  prohibitive  in 
this  program,  the  DFCs  were  estimated  or  decompositions 
simulated.  The  estimation  procedure  attempted  to  predict 
the  DFC  without  performing  a  full  function  decomposition. 

The  simulation  procedure  attempted  to  simulate  a  complete 
function  decomposition.  It  was  not  rigorous,  but  an  attempt 
to  capture  some  of  the  effects  that  occur  while  doing  a 
decomposition.  The  structured  genetic  algorithm  and  the 
baseline  genetic  algorithm  were  evaluated  in  experiments  by 
using  both  the  simulation  and  estimation  procedures.  The 
following  discussion  describes  the  simulation  and  estimation 
procedures  and  the  results  of  all  experiments.  The 
experimental  results  include  comparisons  of  computational 
time  to  find  the  DFC  between  the  two  genetic  algorithms  and 
"chance  without  replacement."  The  ability  to  find  the  DFC 
and  not  some  local  minimum  is  scrutinized.  Also,  a 
comparison  with  chance  was  performed  of  the  ability  of  the 
two  genetic  algorithm  approaches  to  search  increased  numbers 
of  partition  matrices  for  increasing  numbers  of  input 
variables.  The  influence  of  various  genetic  algorithm 
parameters  and  heuristics  were  also  evaluated  in  the 
experiments . 

4 . 1  Estimation  Description 

In  the  estimation  procedure  the  algorithm  would  select 
a  partition  and  evaluate  it  by  performing  a  single  level  of 
decomposition.  It  would  then  estimate  the  cardinality  for 
each  partition  of  the  function,  based  on  column 
multiplicity.  The  cardinality  was  the  sum  of  cardinality  of 
the  subfunctions  carried  out  to  only  one  level.  In  the 
example  in  Section  2.1.2  (Table  2.10)  for  instance, 
functions  g  and  h  would  not  have  been  decomposed  any 
further.  The  resulting  partition  space  would  be  "bumpier" 
(have  more  peaks  and  valleys)  than  if  the  entire 
decomposition  had  been  performed.  For  example,  recall 
Figure  2 . 9  pictured  again  on  the  left  in  Figure  4.1.  The 
partition  space  resulting  from  use  of  the  estimation 
procedure  in  comparison  would  appear  as  shown  on  the  right 
in  Figure  4.1. 
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Figure  4.1.  Comparison  of  Partition  Space  Resulting  from 
Full  Decomposition  (Left)  to  that  Resulting  from  Estimation 

Procedure 


The  estimated  data  could  result  in  what  is  referred  to 
as  a  positive  decomposition.  This  results  if  the  estimated 
cardinality  is  higher  than  the  original  cardinality.  In  the 
algorithm  this  positive  decomposition  could  be  allowed  or 
disregarded  for  experimental  analyses.  If  the  algorithm  was 
set  to  suppress  the  positive  decomposition  then  the  original 
cardinality  was  used  as  the  estimate.  The  estimation 
procedure  was  tried  for  26  functions  in  the  experimental 
analysis.  Also,  for  the  estimation  procedure,  only  eight 
variable  functions  were  used. 

4.2  Simulation  Description 

In  the  simulation  procedure  the  user  decides  which  sets 
of  input  variables  must  be  grouped  in  columns,  in  rows,  or 
can  be  mixed.  The  algorithm  then  attempts  to  find  the 
row/column  assignments.  It  makes  trial  row  and  column 
selections  by  breaking  out  certain  variables  as  subfunctions 
and  calculates  a  value  that  corresponds  to  the  cardinality 
for  each  row/column  selection.  This  calculation  is  based 
upon  the  discussion  in  Section  2.1.1.  The  new  cardinality 
is  compared  to  the  original  cardinality.  Three  particular 
cases  apply  in  the  simulation  procedure  when  making 
row/column  partition  selections. 

CASE  l: 

If  a  set  of  input  variables  is  specified  to  appear 
entirely  in  rows  and  the  algorithm  selects  them  as  such: 
divide  cardinality  by  and  add  2*^  to  the  cardinality 

for  each  input  of  M  variables  where  this  occurs. 

-  Otherwise  the  cardinality  remains  the  same  as  the 
original  cardinality. 

CASE  2: 

If  a  set  of  input  variables  is  specified  to  appear 
entirely  in  columns  and  the  algorithm  selects  them  as  such: 
divide  cardinality  by  2(^“1)  and  add  2^  to  the  cardinality 
for  each  input  of  M  variables  where  this  occurs. 

-  Otherwise  the  cardinality  remains  the  same. 


CASE  3: 


If  a  function  can  appear  partially  in  rows  or  columns 
and  the  algorithm  selects  them  as  such:  divide  cardinality 
by  2(M“2)  and  add  2(^-2)  to  the  cardinality  for  each  input 
of  M  variables  where  this  occurs.  This  is  analogous  to  the 
m*(2M)  +  2(N"’M+m)  example  in  Figure  2.6,  where  m=2. 

-  If  the  function  appears  entirely  in  rows  or  columns 
the  cardinality  is  calculated  by  the  rules  in  Case  2  or  3. 

There  are  other  conditions  that  must  be  satisfied  or  the 
original  cardinality  is  returned: 

1)  At  least  one  variable  must  be  a  row  variable. 

2)  At  least  one  set  of  input  variables  must  be  entirely 
in  columns. 

3)  The  final  cardinality  must  be  less  than  the  original 
cardinality. 


In  the  baseline  genetic  algorithm  positive 
decompositions  could  be  allowed  if  they  occur  or  such 
decompositions  could  be  suppressed.  Positive  decompositions 
were  always  suppressed  when  the  estimated  data  was  used. 
Again,  a  positive  decomposition  refers  to  a  decomposition 
where  the  new  cardinality  is  larger  than  the  original 
cardinality.  With  positive  decomposition  suppressed  no 
cardinality  larger  than  the  original  cardinality  is  used  as 
an  estimate.  No  excursions  were  tried  with  the  crossover 
operator  in  the  baseline  genetic  algorithm. 

In  the  structured  genetic  algorithm  positive 
decompositions  were  also  allowed  for  the  simulated  data. 
Excursions  were  also  tried  with  the  crossover  operator  later 
in  the  program  to  try  to  improve  the  search  and  find  more 
DFCs.  In  some  cases  the  compliment  of  the  initial 
population  or  P[0]  compliment  was  taken  of  the  best 
performers  after  the  initial  search  terminated.  Observation 
of  other  trials  showed  that  sometimes  the  DFC  occurred  at 
the  compliment  of  the  best  cardinality  chromosomes  found. 
These  trials  did  result  in  finding  more  DFCs.  In  some  cases 
all  of  the  partitions  leading  to  the  DFC  were  found. 

Another  crossover  operator  excursion  was  called  the  "odd 
child  fix."  This  excursion  was  tried  in  the  structured 
search  algorithm,  producing  4  children  if  the  number  of 
different  alleles  was  odd  between  two  mating  chromosomes. 
This  excursion  was  used  because  observation  of  a  number  of 
trials  seemed  to  indicate  the  search  will  take  longer  if  all 
possibilities  are  not  examined  initially.  In  this  crossover 
scheme  two  crossover  points  were  selected  as  shown  in  the 
example  below.  The  "odd  child  fix"  improved  speed 
significantly  and  performance  (ability  to  find  the  actual 
DFC)  marginally  on  the  simulated  data.  In  cases  where  there 
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were  an  even  number  of  differences  between 
structured  crossover  was  used. 

the  parents  then 

Parent  A 
Parent  B 

xxxxx 

yyyyy 

Child  C 
Child  D 

after  two  children. 

xxyyy 

yyxxx 

The  old  method 

stopped  here 

Child  E 
Child  F 

xxxyy 

yyyxx 

The  "odd  child 

fix"  added  two 

children. 


4.3  Svimmary  of  Experimental  Results 

Experimental  results  were  analyzed  to  see  if 
improvements  over  "chance  without  replacement"  were  realized 
by  using  the  structured  or  baseline  algorithms.  An 
improvement  was  judged  to  be  a  reduction  in  the  number  of 
partition  matrices  searched  before  the  DEC  was  discovered. 
Appendix  B  is  a  comprehensive  table  listing  all  the 
experimental  results.  Figure  4.2  summarizes  experimental 
data  from  the  performance  comparison  of  the  baseline  and 
structured  genetic  algorithm  approaches.  These  results  are 
taken  from  trials  using  simulated  data  to  identify  the  DFC. 
The  curves  shown  are  averages  from  typical  experiments  using 
a  niimber  of  functions.  The  data  show  the  number  of  initial 
row/column  selections  made  before  the  DFC  was  found  as  a 
function  of  the  number  of  input  variables.  As  Figure  4.2 
shows,  both  the  structured  and  baseline  algorithms  reduce 
the  search  space  compared  to  the  random  or  chance  approach. 
As  the  number  of  input  variables  are  increased,  the 
effectiveness  of  the  genetic  algorithm  approaches  are 
magnified.  The  structured  approach  in  particular 
dramatically  reduces  the  number  of  partition  matrices  that 
are  examined  before  the  DFC  is  found.  The  experimental 
results  showed  that  the  structured  search  algorithm  can  find 
an  answer  up  to  10  times  faster  than  exhaustive  search  for  8 
bit  functions  with  simulated  data.  However,  the  cardinality 
found  is  sometimes  not  the  minimum  cardinality  (DFC) . 
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Figure  4.2.  Improvement  of  Baseline  and  Structured 
Algorithms  Over  Chance  for  Simulated  Data. 


Tables  4.1  to  4.4  provide  a  detailed  representation  of 
the  experimental  results  for  estimated  data.  The 
performance  of  the  baseline  and  structured  algorithms  are 
compared  to  "chance  without  replacement."  A  comparison  can 
also  be  made  between  the  performance  of  the  two  algorithms. 
The  influence  of  allowing  or  suppressing  positive 
decompositions  on  finding  the  DFC  is  examined.  Also  shown 
is  the  effect  of  taking  the  compliment  of  the  initial 
population  and  using  the  "odd  child  fix"  technique  in  the 
structured  algorithm.  Tables  4.1  and  4.2  provide  results 
for  all  the  functions  evaluated  by  each  algorithm.  The  last 
two  tables  present  the  results  for  functions  which  only  have 
one  partition  leading  to  the  DFC.  This  second  comparison 
was  made  to  see  if  the  experimental  results  would  look  any 
different  for  the  most  difficult  searches. 

Tables  4.1  to  4.4  are  actually  two-dimensional  arrays. 
The  columns  show  the  percentage  of  the  time  the  algorithms 
performed  better  or  worse  than  "chance  without  replacement" 
(see  Section  2. 2. 2. 2).  Better  than  chance  means  the  answer 
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was  found  faster  than  by  using  "chance  without  replacement" 
procedures.  The  computational  speed  refers  to  the 
reduction  in  the  partition  space  that  had  to  be  searched  by 
the  new  algorithms  compared  to  "chance  without  replacement." 
For  example,  4x  the  speed  means  that  only  1/4  of  the 
partition  matrices  had  to  be  searched  as  compared  to  an 
exhaustive  search  ("chance  without  replacement") . 

Obviously,  this  reduction  in  search  space  will. also  reduce 
computational  time  significantly.  The  rows  refer  to  the 
ability  to  find  the  actual  DFC.  The  row  marked  "DFC  Found" 
refers  to  the  percentages  when  the  actual  DFC  was  found. 

"DFC  Not  Found"  refers  to  the  percentage  of  time  when 
something  other  than  the  DFC  was  found.  Recall  that  the 
structured  search  algorithm  has  a  termination  condition  that 
can  result  in  the  termination  of  a  search  before  the  DFC  is 
found  ( see  Table  3.4). 


Table  4.1.  Stiructured  Genetic  Algorithm  Performance  All 

Functions . 


With  Positive  Decomposition 


%  Time 

%  Time 

Computational 

Better 

Worse 

Speed 

Than 

Than 

Improvement 

Chance 

Chance 

Over  Chance 

DFC  Found 

41.42 

6.82 

7.14X 

DFC  Not 

51.43 

0.34 

Found 


With  Positive  Decomposition  Suppressed 


%  Time 

%  Time 

Computational 

Better 

Worse 

Speed 

Than 

Than 

Improvement 

Chance 

Chance 

Over  Chance 

DFC  Found 

55.23 

12.34 

4.47X 

DFC  Not 

32.2 

0.23 

Found 

With  Positive 

Decomposition 

and  P[0]  Compliment 

%  Time 

%  Time 

Computational 

Better 

Worse 

Speed 

than 

than 

Improvement 

Chance 

Chance 

Over  Chance 

DFC  Found 

63.80 

3.62 

4 . 4x 

DFC  Not 

32.25 

0.34 

Found 

With  Positive 

Decomposition 

and  "Odd  Child 

Fix" 

%  Time 

%  Time 

Computational 

Better 

Worse 

Speed 

than 

than 

Improvement 

Chance 

Chance 

Over  Chance 

DFC  Found 

64.15 

7.15 

4.29X 

DFC  Not 

28.34 

0.35 

Found 

42 


Table  4.2.  Baseline  Genetic  Algorithm  Performance  All 

Functions . 


With  Positive 

Decomposition 
%  Time 

%  Time 

Computational 

Better 

Worse 

Speed 

than 

than 

Improvement 

Chance 

Chance 

Over  Chance 

DFC  Found 

69.69 

30.31 

1.5x 

DFC  Not 

0.0 

0.0 

Found 

With  Positive 

Decomposition  Suppressed 

%  Time 

%  Time 

Computational 

Better 

Worse 

Speed 

than 

than 

Improvement 

Chance 

Chance 

Over  Chance 

DFC  Found 

71.85 

27.23 

1.56X 

DFC  Not 

0.0 

0.92 

Found 
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Table  4.3.  Structured  Genetic  Algorithm  Performance 
Functions  With  One  Decomposition  or  Partition  Leading  to  the 

DFC. 


With  Positive 

Decomposition 
%  Time 

%  Time 

Computational 

Better 

Worse 

Speed 

than 

than 

Improvement 

Chance 

Chance 

Over  Chance 

DFC  Found 

50.84 

4.51 

11.49X 

DFC  Not 

44.51 

0.15 

Found 

With  Positive 

Decomposition  Suppressed 

%  Time 

%  Time 

Computational 

Better 

Worse 

Speed 

than 

than 

Improvement 

Chance 

Chance 

Over  Chance 

DFC  Found 

58.15 

0.0 

6. 85x 

DFC  Not 

41.75 

0.11 

Found 

With  Positive 

Decomposition 

and  P [ 0 ]  Compliment 

%  Time 

%  Time 

Computational 

Better 

Worse 

Speed 

than 

than 

Improvement 

Chance 

Chance 

Over  Chance 

DFC  Found 

69.13 

0.04 

7.35X 

DFC  Not 

30.58 

0.25 

Found 

With  Positive 

Decomposition 

and  "Odd  Child 

Fix" 

%  Time 

%  Time 

Computational 

Better 

Worse 

Speed 

than 

than 

Improvement 

Chance 

Chance 

Over  Chance 

DFC  Found 

69.13 

0.04 

7.15X 

DFC  Not 

30.65 

0.18 

Found 
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Table  4.4.  Baseline  Genetic  Algorithm  Performance  Functions 
With  One  Decomposition  or  Partition  Leading  to  the  DFC. 


With  Positive 

Decomposition 
%  Time 

%  Time 

Computational 

Better 

Worse 

Speed 

than 

than 

Improvement 

Chance 

Chance 

Over  Chance 

DFC  Found 

87.45 

12.55 

2.19X 

DFC  Not 

0.0 

0.0 

Found 

With  Positive 

Decomposition 

Suppressed 

%  Time 

%  Time 

Computational 

Better 

Worse 

Speed 

than 

than 

Improvement 

Chance 

Chance 

Over  Chance 

DFC  Found 

82.73 

16.55 

2 . 25x 

DFC  Not 

0.0 

0.73 

Found 

These  tables  show  several  significant  findings  from  the 
genetic  algorithm  performance  comparison.  In  the  structured 
algorithm  allowing  positive  decompositions  led  to  increased 
computational  speed  over  suppression  of  positive 
decompositions,  but  the  algorithm  did  not  locate  the  DFC  as 
well.  Adding  the  P[0]  compliment  or  "odd  child  fix" 
routines  slowed  down  the  computational  speed,  but  improved 
the  chance  of  finding  the  DFC.  In  the  baseline  search 
positive  decompositions  did  not  make  a  drastic  difference. 
Computational  speed  improvement  over  chance  was  still 
significant  for  the  baseline  algorithm,  but  not  as  much  as 
the  structured  search.  However,  the  baseline  algorithm 
found  the  DFC  virtually  every  time,  while  the  structured 
search  located  local  minima  a  fair  amount  of  the  time.  For 
functions  with  only  one  partition  leading  to  the  DFC,  the 
structured  and  baseline  genetic  algorithms  performed  even 
better,  compared  to  chance,  than  for  functions  with  multiple 
DFCs.  In  other  words,  the  more  difficult  the  search  the 
more  benefit  the  genetic  algorithms  could  provide.  The 
structured  approach  showed  less  dependence  on  the  type  of 
function . 

Recall  that  the  DFCs  were  identified  in  this 
experimental  program  by  either  a  simulation  procedure  or 
estimation  procedure,  they  were  not  located  by  performing 
full  function  decompositions.  Since  there  were  differences 
between  these  two  procedures,  the  performance  comparison  of 
the  algorithms  showed  dependence  on  the  particular  procedure 
used  to  identify  the  DFC.  Table  4.5  summarizes  some  of  the 
findings  from  the  algorithm  performance  comparison  in 
relation  to  the  estimation  or  simulation  procedures.  Both 
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algorithms  were  an  improvement  over  chance  in  the  time 
required  to  locate  the  DFC,  whether  the  estimation  or 
simulation  procedure  was  used.  However,  as  the  table 
states,  both  algorithms  sometimes  identified  a  local  minimum 
rather  the  DFC  for  the  function.  This  occurred  much  more 
often  using  the  estimation  procedure  or  estimated  data. 

Using  the  simulated  data,  both  algorithms  were  much  more  apt 
to  locate  the  actual  DFC.  The  table  also  shows  that  the 
baseline  algorithm  was  better  able  to  locate  the  actual  DFC 
when  compared  to  the  structured  algorithm.  This  was  true 
whether  the  estimated  data  or  simulated  data  was  used. 


Table  4.5.  Performance  Comparison  of  Baseline  and 
Structured  Genetic  Algorithm  Approaches. 


BASELINE 

STRUCTURED 

Speed  improvement  over 
chance . 

Speed  improvement  over 
chance.  Less  function 
dependent  variance. 

Rarely  gets  stuck  on  a 
local  minimum  using 
simulated  data. 

Rarely  gets  stuck  on  a 
local  minimum  using 
simulated  data,  but  gets 
stuck  more  often  than 
baseline. 

Often  gets  stuck  on  a  local 
minimum  using  estimated 
data. 

Gets  stuck  more  often  than 
baseline  on  a  local  minimum 
using  estimated  data. 
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4.4  Analysis  of  Heuristic/Parameter  Variations 

Although  an  exhaustive  parametric  study  was  not 
conducted,  variations  in  some  of  the  genetic  algorithm 
parameters  and  heuristics  did  allow  some  conclusions  to  be 
assessed  from  the  experimental  results.  Many  of  the 
parameters  and  heuristics  were  defined  differently  between 
the  baseline  and  structured  genetic  algorithm  approaches, 
making  their  influence  often  apparent.  Table  4.6  describes 
the  different  parameters  and  heuristics  evaluated  for  each 
of  the  two  algorithms.  It  also  shows  which  parameters  were 
examined  in  detail  and  which  were  tried,  but  for  one  reason 
or  another  were  not  used  in  the  experiments.  The  initial 
population  was  either  large,  using  many  chromosomes,  or 
limited,  starting  out  with  fewer  chromosomes.  The  crossover 
operators  used  were  described  previously.  Mutation  was  only 
used  for  the  baseline  algorithm  in  some  cases.  Reproduction 
rules  were  used  as  described  earlier.  Tailored  probability 
refers  to  the  selection  strategy  shown  in  Figure  3.1. 

"Best”  refers  to  cases  where  only  the  best  performers  at  the 
front  of  the  sorted  population  array  were  reproduced.  "N- 
best"  refers  to  cases  where  some  of  the  best  lower 
performing  chromosomes,  but  not  the  worst  performers,  were 
allowed  to  reproduce.  Real  uniform  refers  to  cases  where 
chromosomes  were  selected  uniformly  throughout  the 
population  for  reproduction.  Table  4.7  illustrates  the 
influences  of  these  different  parameter  and  heuristic 
settings  on  the  performance  of  each  algorithm. 
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Table  4.6.  Key  Genetic  Algorithm  Parameters  and  Heuristics 

Analyzed  in  Experiments. 


Initial  P 

Baseline 

opulation 

Structured 

Few 

Many 

nmy 

Cross 

Baseline 

30ver 

Structured 

Single 

Structured 

Single 

Double 

Double 

Higher 

Order 

Higher 

Order 

Muta 

Baseline 

tion 

Structured 

None 

■  ■ 

Rate 

Mat 

Bass 

Struc 

Uniform 

ing 

sline 

tured 

Ordered 

•  Tailored 
'ProbaMldt; 

:• . ■ . t. . 

: 

N-best 

N-best 

Real 

Uniform 

Legend 


Not  Tried 


Tried,  But  Not 
Used 

Used  For  Data  In 
Experiment 


>8 


Table  4.7.  Influence  of  Key  Genetic  Algorithm  Parameters 


Crossover 

o  Uniform  crossover  tends  to  work  best 
&  fastest.  It  worked  best  in  baseline 
search. 

o  Single  or  double  crossover  takes 
longer.  Single  crossover  used  with 
structured  search  tends  to  increase 
time  in  baseline  search, 
o  Using  the  structured  crds sever 
operator  with  the  baseline  search  to 
generate  an  initial  population  results 
in  the  search  getting  stuck  in  a  local 
minimum . 

Mutation 

o  If  mutation  is  not  used  searches 
will  get  stuck  in  a  local  minimum  as 
literature  predicts.  Trying  some 
mutation  in  structured  search  might 
help. 

Selection  Strategy 
for  Reproduction 

o  Tailored  probability  seemed  to  work 
best  in  baseline  search, 
o  Limiting  mating  to  N-best  in 
baseline  search  caused  some  local 
minimum  problems. 

o  Using  something  other  than  tailored 
probability  for  choosing  parents  in 
the  baseline  search  resulted  in  a 
longer  search. 

o  In  the  structured  search  mating 
using  N-best  (N>1)  relieved  some  of 
the  local  minimum  problems,  but  caused 
it  to  take  much  longer. 

49 


5.0  Conclusions 


5.1  Conclusions  from  Experiments 

The  experiments  conducted  in  this  program  on  the 
baseline  and  structured  genetic  search  algorithms  showed 
very  promising  results.  Both  algorithms  performed  faster 
than  chance  overall.  This  is  significant  since  it  points 
out  that  elements  of  both  algorithms  can  be  used  to  reduce 
computational  time  for  function  decomposition.  Speed  is 
important,  but  the  ability  to  find  the  DEC  is  vital.  Both 
algorithms  showed  the  ability  to  find  the  DFC  and  yet  be 
faster  than  chance.  Experiments  showed  that  the  structured 
search  was  faster  than  the  baseline,  but  did  not  always  find 
the  DFC  using  estimated  data,  because  of  a  restrictive 
reproduction  and  termination  condition.  The  structured 
search,  however,  always  found  the  DFC  using  the  simulated 
data.  A  terminating  condition  that  needs  to  be  included  in 
the  baseline  algorithm  will  likely  reduce  some  of  its 
ability  to  find  the  DFC.  The  accurate  performance  of  these 
algorithms  for  various  combinations  of  heuristics  and 
parameter  settings,  along  with  an  increase  in  speed  over 
chance,  warrants  a  further  investigation  of  their 
applicability. 

The  structured  search  algorithm  can  be  tailored  to 
include  additional  heuristics  that  could  improve  its 
performance.  Several  excursions  were  tried  in  this  program, 
as  mentioned  previously,  such  as  allowing  mating  of  slightly 
"less  fit"  decompositions.  These  excursions,  however,  were 
found  to  only  increase  the  time  required  for  convergence  in 
the  simulated  data.  They  were  not  tried  on  the  estimated 
data. 


Both  algorithms  showed  improvement  over  chance  in  the 
nximber  of  partition  matrices  searched,  as  the  number  of 
variables  increased.  The  rate  of  growth  of  the  search  space 
is  still  exponential,  but  has  a  slower  rate  of  increase. 

This  is  very  promising,  since  as  the  number  of  variables 
increases,  the  time  required  to  evaluate  a  partition  also 
increases  exponentially. 

Regrettably,  time  did  not  allow  a  full  parametric 
analysis  of  the  genetic  algorithm  approaches.  However,  the 
work  performed  up  to  this  point  looks  promising.  Both 
algorithms  can  provide  a  significant  speed  increase  in 
finding  the  DFC.  Additional  improvements  can  probably  be 
attained  by  using  a  better  method  of  selecting  the  initial 
population,  i.e.,  correlation  techniques  and/or  increasing 
the  number  of  rows  in  the  initial  population  (see  Section 
2.2.2).  Both  algorithms  would  need  to  be  tailored  depending 
on  the  application. 
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5.2  Recommendations/ Future  Work 

For  limitations  that  were  found,  several  possible  fixes 
could  be  studied  to  potentially  solve  the  problems.  Other 
genetic  algorithm  variations  could  be  investigated, 
including  both  standard  and  nonstandard  practices.  The 
particular  problems  that  were  encountered  could  be  studied 
more  to  identify  particular  heuristics  or  theories  that  may 
prove  useful.  Some  were  tried  in  this  program  already. 
Additional  work  could  identify  better  ways  to  generate  the 
initial  population.  The  P[0]  compliment,  for  example,  could 
be  applied  to  more  of  best  performers  than  just  the  top 
ones. 


Some  of  the  open  issues  that  may  require  additional 
investigation  include  the  presence  of  a  large  number  of 
discontinuities  in  the  estimated  data.  The  estimated  data 
may  not  have  provided  a  very  accurate  estimate  of  the  DFC. 
Obviously,  it  would  be  beneficial  to  try  both  algorithms  on 
actual  data,  or  in  other  words  allow  a  function  to  be 
completely  decomposed  to  the  DFC.  A  comparison  could  then 
be  made  between  the  actual  data  and  the  estimated  and 
simulated  data.  A  program  called  FLASH  (Function  Learning 
and  Synthesis  Hotbench)  resides  at  the  Avionics  Directorate 
that  carries  out  decompositions  more  thoroughly  than  those 
algorithms  used  in  this  study.  This  program  could  be 
applied  to  functions  examined  in  this  study  to  verify  the 
effectiveness  of  the  structured  and  baseline  algorithms. 
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Appendix  A. 

Correlation  Partition  Selection  Algorithm  Paper 

Michael  J.  Noviskey 


30  Aug  92 

1.0  Introduction 

This  memo  documents  work  done  on  a  partition  selection  algorithm 
for  function  decomposition  as  described  in  Ref  1.  This  memo 
includes  a  brief  description  of  the  need  for  partition  selection 
algorithms;  a  candidate  algorithm  for  generation  of  coefficients 
corresponding  to  the  input  variables  of  a  function;  how  to  use 
the  coefficients  in  selecting  rows  and  columns  of  a  partition 
matrix;  and  several  examples  and  observations  about  this 
algorithm. 

2.0  The  need  for  partition  selection 

In  order  to  decompose  a  function  the  Ada  Function  Decomposition 
(AFD)  algorithms  essentially  perform  an  exhaustive  search,  with  a 
few  limiting  heuristics.  There  are  about  two  raised  to  the 
number  of  input  variables  partition  matrices  to  be  searched  for 
the  original  function.  If  the  function  decomposes,  more 
functions  of  lower  complexity  are  produced  for  further 
decomposition.  This  process  is  continued  until  functions  of  the 
lowest  complexity  are  found.  No  attempt  was  made  to  select 
partitions  that  would  decompose  to  some  low  cost  initially.  If 
this  were  done  it  could  limit  the  search  space  or  prune  the 
search  tree  later  on.  No  attempt  was  made  to  develop  heuristics 
which  would  predict  good  partitions  for  decomposition.  These 
choices  were  made  in  order  to  meet  some  of  the  objectives  of 
Pattern  Theory  I.  But  in  order  to  develop  a  practical  algorithm, 
some  type  of  partition  selection  algorithm  which  can  guide  the 
search  needs  to  be  developed. 

2.1  Algorithm  description 

The  algorithm  description  is  divided  into  two  main  parts.  The 
first  is  a  description  of  generation  of  coefficients  which 
correspond  to  input  variables.  The  second  is  utilization  of  the 
coefficients  to  select  row  and  column  variables  for  the  partition 
matrix. 

2.1.1  Coefficient  Generation 

For  each  defined  value  of  the  function  a  coefficient  for  each 
input  variable  is  incremented  if  the  value  of  the  input  variable 
matches  the  value  of  the  function.  The  coefficient  is 
decremented  if  the  input  variable  does  not  match  the  value  of  the 
function.  The  following  BASIC  subroutines  provide  a  detailed 
implementation.  These  routines  assume  that  true  vacuous 
variables  have  been  removed.  If  vacuous  variables  are  not 


53 


explicitly  removed  as  a  preliminary  step,  they  will  increase  the 
run  time  by  increasing  the  size  and  number  of  partition  matrices 
which  must  be  considered. 

Two  routines  not  documented  here  are  ZeroVector  and  GetBinary. 
ZeroVector  sets  the  elements  of  a  one  dimensional  array  to  zero. 
And  GetBinary  returns  a  binary  vector  representation  of  a  base 
ten  argument.  In  this  case  the  vector  returned  from  GetBinary 
represents  the  input  to  the  function. 

SUB  GenCoefs  (Function$,  A%(),  X%()) 

I 

'  The  purpose  of  this  routine  is  to  generate  partition 
selection 

'  coefficients. 

I 

'  Function$  =>  A  string  representation  of  a  Function. 

I 

'  X$  =>  The  Nth  value  of  the  function 

'  {0,1, X} 

I 

'  X%()  =>  Binary  vector  representation  of  input 

for  the 

'  Nth  value  of  the  function. 

'  Vector  of  {0,1} 

I 

’  A%()  =>  Partition  selection  coefficients. 

'  Vector  of  integers 

ZeroVector  A%():  •  First  set  partition 

selection 

'  coefficients  to  zero. 

FOR  N%  =  1  TO  LEN(Function$) :  '  For  each  element  of 

the  function: 

X$  =  MID$(Function$,  N%,  1):  »  Get  it's  value. 

'  If  this  element  of 

the  function 

'  is  defined  {0,1}. 

IF  X$  =  "0"  OR  X$  =  "1"  THEN 

GetBinary  (N%  -  l),  x%():  '  Get  it's  binary 

input  vector. 

GenCoef  VAL(X$) ,  A%(),  X%():  '  Modify  the 

Coefficients. 

ELSE 

'  If  this  element  of 

the  function 

'  is  not  defined  then 

do  nothing. 

END  IF 
NEXT  N% 

END  SUB 
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SUB  GenCoef  (Value%,  A%(),  X%()) 


The  purpose  of  this  routine  is  to  increment  or 

decrement 

partition  selection  coefficients  depending  on  whether  the 
associated  input  matches  the  Nth  Value  of  the  function. 


the 


Value%  => 
X%()  => 

A%()  => 


The  Nth  value  of  the  function 

{0,1} 

Binary  vector  representation  of  input  for 

Nth  value  of  the  function. 

Vector  of  {0,1} 

Partition  selection  coefficients. 

Vector  of  Integers 


For  each  input: 

FOR  1%  =  LBOUND(X%)  TO  UBOUND(X%) 


value 


If  it  matches  the 
of  the  function 


IF  X%(I%) 
A%(I%) 

coefficient; 

ELSE  : 
A%(I%) 

coefficient. 

END  IF 
NEXT  1% 

END  SUB 


Value%  THEN 
A%(I%)  +  l: 

A%(I%)  -  1; 


'  increment  it's 

'  otherwise 
'  decrement  it ' s 


2.1.2  Coefficient  Utilization 


The  above  procedure  will  produce  coefficients  associated  with  the 
input  variables.  Selecting  groups  of  coefficients  of  the  same 
magnitudes  as  column  or  row  variables  can  result  in  a  good  choice 
for  a  partition  matrix.  Since  coefficients  of  the  same  absolute 
value  may  or  may  not  result  in  a  decomposition,  combinations 
within  a  group  must  also  be  tried.  However,  combinations  across 
groups  do  not  appear  to  need  to  be  tried.  At  least  no  case  has 
been  found,  where  combinations  across  groups  resulted  in  a 
decomposition . 
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To  illustrate  this  procedure,  suppose  we  have  three  sets  of 
coefficients  al,  a2,  a3;  bl,  b2;  and  cl.  Where  al,  a2,  and  a3 
are  of  the  same  absolute  value;  bl  and  b2  are  of  the  same 
absolute  value.  The  following  combinations  must  be  tried  as 
column  variables,  again  assuming  vacuous  variables  have  been 
removed.  Combinations  which  are  tried  as  coliimn  choices  are  shown 
between  the  []  brackets. 


[(al,  a2,  a3),  (bl,  b2)],  [(al,  a2,  a3) ,  (bl) ] , 

[(al,  a2,  a3),  (b2)], 

[(al,  a2,  a3),  (cl)],  [(al,  a2,  a3)], 

[(al,  a2),  (bl,  b2),  (cl)], 

[(al,  a2,  a3),  (bl) ,  (cl)],  [(al,  a2,  a3) ,  (b2) ,  (cl)], 

[(al,  a3),  (bl,  b2),  (cl)],  [(a2,  a3) ,  (bl,  b2) ,  (cl)], 

[(al,  a2),  (bl,  b2)],  [(al,  a3) ,  (bl,  b2)], 

[(a2,  a3),  (bl,  b2)], 

[(al,  a2),  (cl)],  [(al,  a3),  (cl)),  ((a2,  a3),  (cl)], 
[(al),(bl,  b2)),  ((a2),(bl,  b2)),((a3),  (bl,  b2) ] , 
[(al),(cl)], 

[(a2),(cl)],  [(a3),  (cl)],  [(bl,  b2) ,  (cl)],  [ (bl) ,  (cl)], 
[(b2),  (cl)],  and  [(bl,  b2)]. 

Notice  that  combinations  like  ((al,  a2),  (bl) )  were  not  tried, 
since  no  case  has  been  found  were  choosing  some  elements  from  one 
group  and  some  from  another  results  in  a  good  decomposition. 

This  results  in  twenty  six  rather  than  fifty  six  partitions  (not 
testing  for  vacuous  variables)  which  would  have  been  required  to 
perform  an  exhaustive  search  of  a  six  variable  function  .  The 
percent  reduction  in  search  space  is  more  dramatic  as  the  number 
of  variables  increases,  but  it  depends  on  the  number  of  groupings 
and  number  of  elements  within  a  group.  The  worst  case  occurs 
either  when  all  the  coefficients  are  the  same  or  all  are 
different.  Then  the  entire  space  must  be  searched  again.  The 
best  case  occurs  when  the  number  of  groups  is  about  equal  to  the 
nxamber  of  elements  in  the  groups.  The  treatment  of  the  actual 
number  of  partition  matrices  which  must  be  tried  as  a  function  of 
number  of  groupings  and  grouping  size  is  a  little  more  complex 
and  still  needs  to  be  addressed. 


2 . 2  Examples 

2.2.1  A  very  simple  example 

This  example  illustrates  the  column  selection  procedure  on  a  very 
simple  function  of  three  variables.  Initially  Al,  A2  and  A3  are 
incremented,  since  XI,  X2  and  X3  all  match  the  first  value  of  the 
function.  Al  and  A2  are  incremented  again  since  XI  and  X2  match 
the  second  value  of  the  function,  but  A3  is  decremented  since  X3 
does  not  match  the  function  value.  This  process  is  repeated  for 
the  entire  function  until  the  final  values  2,  6  and  2  are  found 
Al,  A2  and  A3,  (see  figure  below).  Since  the  coefficients 
for  Al  and  A3  have  the  same  magnitudes,  this  suggests  that  XI  and 
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X3  should  be  grouped.  Below  are  three  of  the  partition  matrices 
which  can  be  formed  from  the  function.  Note  that  when  XI  and  X3 
are  selected  as  column  variables,  column  multiplicity,  Nu,  is 
minimized.  This  function 

XI  X2  X3  F  A1  A2  A3 
0  0  0  0  111 

0  0  1  0  2  2  0 

0  1  0  0  3  1  1 

011  1  222 

1  0  0  0  1  3  3 

1  0  1  0  0  4  2 

110  1  15  1 

111  1  262 

could  be  broken  down  into  FI  (  F2  (  XI,  X3) ,  X2) ,  although  no 
cost  savings  would  occur  for  a  three  variable  function. 

X2  &  X3  XI  &  X3  XI  &  X2 

XI  0001  X20000  X30001 

0011  0111  0101 

Nu  =  3  Nu  =  2  Nu  =  3 

2.2.2  A  Non  Decomposable  Function 

The  partition  matrices  shown  for  the  four  variable  function  shows 
that  it  will  not  decompose.  Yet  the  partition  selection 
algorithm  will  generate  different  coefficients. 

XI  &  X2  XI  &  X3  XI  &  X4 


X3 

1  0 

1 

1 

X2 

1 

0 

1  0 

X2 

10  0  0 

& 

1  1 

0 

1 

& 

1 

1 

0  1 

Sc 

110  0 

X4 

0  0 

0 

0 

X4 

0 

0 

1  1 

X3 

1111 

1  1 

1 

0 

0 

0 

1  0 

0  110 

XI 

& 

X2  & 

X3 

XI 

& 

X2 

Sc  X4 

X4 

1  0 

1 

111 

0 

1 

X3 

1 

0 

1 

1 

0  0  0  0 

0  0 

0 

oil 

1 

0 

1 

1 

0 

1 

1110 

XI 

& 

X3  & 

X4 

X2 

Sc 

X3 

&  X4 

X2 

1  0 

1 

10  0 

1 

1 

XI 

1 

1 

1 

0 

0  0  11 

1  1 

0 

10  0 

1 

0 

0 

1 

0 

1 

0  0  10 

The  coefficients  generated  by  the  partition  selection  algorithm 
were  -2,  -2,  6,  -6  respectively.  This  suggests  that  the  first 
matrix  or  one  of  the  last  four  would  give  a  good  decomposition. 
Yet  no  decomposition  works. 
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2.2.3  Absolute  vs  Signed  Values  Consider  the  function; 


((XI  AND  (NOT  X2))  OR  (X3  AND  (NOT  X4) ) ) 

Which  has  partition  selection  coefficients  -6,  6,  -6,  6.  If 
partition  selection  were  based  on  signed  value  then  the  following 
matrices  would  be  tried. 


XI  &  X3 

X2  0  0  0  0 
&  10  10 
X4  1100 
1110 


XI 

& 

X2  St  X3 

XI 

St 

X2  St  X4 

X4 

0  0 

0 

0  10  1 

0 

X3 

0  0 

0 

0  110 

0 

1  1 

0 

0  111 

0 

0  1 

0 

1111 

0 

XI 

St 

X3  St  X4 

X2 

St 

X3  St  X4 

X2 

0  0 

1 

0  111 

1 

XI 

0  0 

1 

110  1 

1 

0  0 

1 

0  0  0  1 

0 

0  0 

0 

0  10  1 

0 

These  would  not  be  tried. 


XI  &  X2 


XI  &  X4 


X3  0010 
&  0  0  10 
X4  1111 
0  0  10 


X2  0011 
&  10  11 
X3  0000 
10  10 


If  signed  values  were  used  instead  of  absolute  values  the  optimal 
decomposition  would  have  been  missed.  Use  of  absolute  values  of 
the  coefficients,  however,  increases  the  search  space  since  more 
coefficients  of  the  same  absolute  value  are  likely  to  exist.  In 
this  rather  simple  example  all  possible  partition  matrices  would 
have  been  tried.  Sign  changes  can  result  from  negation  of  a 
variable  or  a  function  (i.e.,  NOT (OR)  =>  NAND) . 


2.2.4  A  Final  Example 


An  attempt  was  made  to  cause  the  partition  selection  algorithm  to 
fail  by  "designing”  a  function  which  would  force  an  incorrect 
grouping.  This  was  done  by  combining  the  same  function  on 
different  sets  of  input  variables  with  an  OR  gate.  Since  each 
subfunction  would  have  the  same  coefficients,  and  simply 
combining  them  with  an  OR  gate  would  cause  this  to  be  transferred 
to  the  output  causing  an  incorrect  grouping. 
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Choose  an  arbitrary  function: 


G(x,  y,  z)  =  (X  AND  (y  OR  z)) 

Combine  two  of  these  with  an  OR  gate: 

F(X1,  X6)  =  G(X1,  X2,  X3)  OR  G(X4,  X5,  X6) 

The  coefficients  generated  for  this  function  are  30,  10,  10,  30, 
10  and  10  respectively.  This  suggests  grouping  XI  and  X4;  and  X2, 
X3,  X5,  and  X6.  This  appears  to  be  an  the  desired  incorrect 
grouping,  since  the  intuitively  correct  groupings  would  have  been 
XI,  X2  and  X3;  and  X4,  X5  and  X6. 

However,  if  the  partition  is  formed: 

X2,  X3,  X5  &  X6 

XI  0000000000000000 
&  0111011101110111 
X4  0000111111111111 
0111111111111111 

0111011101110111  =>X7 

0000111111111111  =>X8 

Further  decompositions  result. 

F  X7  X8 


XI  &  X7 

X2 

&  X3 

X2 

&  X3 

X4 

0  0  0  1 

X5 

0 

111 

X5 

0 

0  0  0 

& 

0  0  0  1 

& 

0 

111 

& 

1 

111 

X8 

0  0  0  1 

X6 

0 

111 

X6 

1 

111 

1111 

0 

111 

1 

111 

Notice  that  X7  is  vacuous  in  X5  and  X6;  X8  is  vacuous  in  X2  and 
X3.  In  fact: 


and 


X7  =  X2  OR  X3, 

X8  =  X5  OR  X6, 

F  =  (XI  AND  X7)  OR  (X4  AND  X8) 


which  was  the  original  function. 

Several  more  experiments  like  this  were  performed  using  AND  and 
XOR  gates  and  more  variables  with  similar  results.  An  initial 
apparently  incorrect  grouping  of  subfunctions  decomposed  to  the 
original  function. 


3 . 0  Summary 

The  procedure  for  calculating  coefficients  has  been  implemented 
in  a  BASIC  program.  The  procedure  for  selecting  partitions  has 
not  yet  been  automated.  Because  of  the  way  the  algorithm  selects 
coefficients  at  various  levels  of  decomposition,  it  needs  to  be 
implemented  in  a  program  which  completes  the  decomposition. 
Examining  initial  groupings  does  not  mean  improperly  selected 
groupings  will  not  be  eliminated  at  a  later  stage  of 
decomposition  via  generation  of  vacuous  variables  (See  example 
2.2.4)  . 


The  partition  selection  algorithm  described  seems  to  be  a  good 
method  for  reducing  the  search  space  for  functional  decomposition 
programs  like  AFD.  There  is  no  theoretical  basis  for  this  method; 
this  needs  development.  It  is  yet  not  known  if  it  covers  the 
entire  search  space  needed  for  optimal  decompositions,  since  at 
this  time  only  a  few  hand  done,  machine  assisted,  examples  have 
been  tried.  Even  if  it  does  not  provide  complete  coverage  of  the 
search  space,  it  usually  provides  a  low  cost  decomposition  and 
could  be  a  good  starting  point  for  limiting  the  search  tree  via 
pruning  techniques. 

There  are  still  a  number  of  questions  about  this  technique.  Does 
it  cover  the  entire  required  search  space?  In  what  order  should 
grouping  be  tried?  How  well  does  it  work  with  partial  functions? 
Only  a  few  examples  have  been  tried,  but  the  technique  seems 
promising.  Further  experimentation  is  needed.  Incorporating 
this  algorithm  in  the  Function  Learning  Synthesis  HotBench  the 
C++  version  of  AFD  is  required  to  answer  these  and  other 
questions . 
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Appendix  B. 

Experimental  Data 


Summary  of  data:  The  following  pages  give  a  summary  of 
the  experimental  runs  for  actual  data. 

Key: 

Baseline  /  structured  /  structured  PIO]  compliment  /  structured  odd  child 
fix  -  refers  to  the  particular  algorithm  used. 

all  functions  -  more  than  one  partition  selection  leads  to  an  optimal  decomposition 

functions  with  one  decomposition  -  only  one  partition  selection  leads  to  an  optimal  decomposition 


Negative  decomposition  suppressed  /  Negative  decomposition  allowed  -  if  values  greater  than  table 
size  were  allowed  in  the  fitness  factor. 

DFC  Found  /  DFC  not  Found  -  number  of  functions  in  which  the  algorithm  found  or  did  not  find  the 
optimal  partition. 

faster  /  slower  -  number  of  cases  the  algorithm  ran  slower  or  faster  than  a  random  search, 
chance  /  actual  -  expected  number  of  partitions  searched  randomly  before  an  optimal  partition  is 
found  and  number  of  average  number  of  partitions  actually  searched  by  the  genetic  algoithm  before 
it  terminated. 

hits  •  number  of  optimal  partitions  in  the  function. 

improvement  -  the  actual  improvement  factor  over  chance  (summed  over  runs  -  not  average) 


Baseline 


all 

functions 


Negative 

decomposition 

suppressed 


DFC 

DFC  not 

chance 

actual 

hits 

improvement 

Found 

Found 

faster 

slower 

faster  slower 

18 

24 

8 

3 

3.43 

21 

0.74 

34 

16 

5.72 

5.53 

3 

1.14 

28 

22 

3.9 

4-1 

12 

0.87 

29 

21 

5.72 

5.86 

3 

0.91 

47 

3 

7 

5.3 

1 

3.25 

38 

12 

5.72 

5.39 

3 

1.26 

41 

9 

5.35 

4.52 

4 

1.78 

38 

12 

5 

4.5 

5 

1.41 

19 

31 

3.1 

4 

20 

0.54 

26 

24 

2.58 

4.44 

15 

0.28 

50 

0 

7 

5.39 

1 

3.05 

44 

6 

7 

6-27 

1 

1.66 

22 

28 

7 

7.14 

1 

0.91 

32 

18 

4.39 

4.49 

8 

0.93 

37 

13 

5.3 

4.8 

4 

1.41 

22 

28 

4.58 

5.31 

7 

0.60 

38 

12 

1 

1.5 

126 

0.71 

41 

9 

6.22 

5.34 

2 

1.84 

38 

12 

6.23 

5.79 

2 

1.36 

50 

0 

7 

5.38 

1 

3.07 

50 

0 

7 

5.14 

1 

3.63 

50 

0 

7 

5.3 

1 

3-25 

41 

9 

7 

6.26 

1 

1.67 

40 

10 

7 

6.26 

1 

1.67 

40 

9 

1 

7 

6.23 

1 

1.71 

21 

26 

3 

7 

7.22 

1 

0.86 

934 

354 

12 

1300 

40,50 

1.56 

times 

better 

71.85% 

27-23% 

0.00%  0.92% 

than 

chance 
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Baseline 


all 

functions 


Negative 

decoinposition 

allowed 

DFC  DFC  not  chance  actual  hits  improvment 

Found  Found 

faster  slower  faster  slower 


17 

33 

3 

3.49 

21 

0.71 

30 

20 

5.72 

5.6 

3 

1.09 

24 

26 

3.9 

4.42 

12 

0.70 

36 

14 

5.72 

5.4 

3 

1.25 

49 

1 

7 

5.89 

1 

2.16 

38 

12 

5.72 

5,2 

3 

1.43 

33 

17 

5.35 

5.1 

4 

1.19 

26 

24 

5 

5.24 

5 

0.85 

5 

45 

3.1 

4.36 

20 

0.42 

17 

33 

3.5 

3.15 

15 

1.27 

49 

1 

7 

5-89 

1 

2.16 

49 

1 

7 

5.89 

1 

2.16 

20 

30 

7 

7.11 

1 

0.93 

30 

20 

4.39 

4.85 

8 

0.73 

31 

19 

5.3 

5.39 

4 

0.94 

21 

29 

4.58 

5.11 

7 

0.69 

36 

14 

1 

1.77 

126 

0.59 

44 

6 

6.22 

5.43 

2 

1.73 

37 

13 

6.23 

5.76 

2 

1.39 

50 

0 

7 

5.33 

1 

3.18 

49 

1 

7 

5.58 

1 

2.68 

50 

0 

7 

5-39 

1 

3.05 

47 

3 

7 

6.13 

1 

1.83 

50 

0 

7 

5.17 

1 

3.56 

38 

12 

7 

6.57 

1 

1.35 

30 

20 

7 

6.9 

1 

1.07 

906 

394 

0 

1300 

39.08 

1.50 

times 

better 

69.69% 

30.31% 

0,00% 

0.00% 

than 

chance 
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structured 


all 

functions 


Negative 

decomposition 

suppressed 


DFC 

DFC  not 

chance 

actual 

hits 

improvment 

Found 

Found 

faster 

slower 

faster 

slower 

131 

119 

3 

4.36 

21 

0.39 

105 

52 

93 

5.72 

3-11 

3 

6.11 

205 

1 

44 

3.9 

2.9 

12 

2.00 

48 

0 

202 

5.72 

3.3 

3 

5.35 

153 

0 

97 

7 

3.73 

1 

9.65 

77 

173 

5.72 

3.28 

3 

5.43 

216 

34 

5.3 

3.48 

4 

3.53 

115 

2 

132 

1 

5 

3.07 

5 

3.81 

133 

117 

3.1 

3.62 

20 

0.70 

128 

122 

3.58 

3.99 

15 

0.75 

250 

0 

0 

0 

7 

4.27 

1 

6.63 

13 

0 

237 

7 

4.38 

1 

6.15 

19 

0 

231 

7 

4.44 

1 

5.90 

159 

6 

78 

7 

4.39 

3.38 

8 

2.01 

131 

1 

114 

4 

5.3 

3.81 

4 

2.81 

123 

127 

4.58 

5.03 

7 

0.73 

186 

64 

1 

0.8 

126 

1.15 

78 

0 

172 

6.22 

4.26 

2 

3.89 

156 

18 

76 

6.23 

5.06 

2 

2-25 

250 

0 

7 

4.63 

1 

5.17 

214 

0 

36 

7 

3.79 

1 

9.25 

210 

0 

40 

7 

3.72 

1 

9.71 

137 

0 

113 

7 

4.3 

1 

6.50 

216 

0 

34 

7 

3.74 

1 

9.58 

106 

0 

144 

7 

4.98 

1 

4.06 

31 

0 

216 

3 

7 

5.56 

1 

2.71 

3590 

802 

2093 

15 

6500 

116.22 

percent 

4.47  times 

better 

55.23% 

12.34% 

32.20% 

0.23% 

than 

chance 

64 


structured  all 

functions 

Negative 

decomposition 

allowed 

DFC  DFC  not  chance  actual  hits  improvment 


Found  Found 


faster 

slower  faster 

slower 

118 

101 

31 

107 

33 

110 

154 

1 

95 

47 

0 

202 

1 

127 

0 

123 

51 

0 

199 

112 

138 

0 

41 

8 

188 

13 

72 

53 

125 

70 

56 

123 

1 

250 

0 

0 

0 

98 

0 

152 

2 

0 

248 

68 

6 

175 

1 

76 

8 

166 

0 

26 

23 

199 

2 

175 

0 

75 

67 

0 

183 

0 

127 

15 

108 

250 

0 

64 

0 

186 

184 

0 

66 

184 

0 

66 

170 

0 

80 

41 

0 

209 

11 

1 

234 

4 

2692 

443 

3343 

22 

percent 

41.42% 

6.82% 

51 .43% 

0.34% 

3 

4.26 

21 

0.42 

5.72 

4.9 

3 

1.77 

3-9 

2.6 

12 

2.46 

5-72 

2.8 

3 
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Summary  of  data:  The  following  pages  give  a  summary  of 
the  experimental  runs  for  simulated  data. 

Key: 

baseline  /  structured  -  refers  to  the  particular  algorithm  used, 
groups  -  the  number  of  groups  inputs  are  devided  into  for  row  column  selection 
hits  -  number  of  optimal  partitions  in  the  function. 

improvement  -  the  actual  improvement  factor  over  chance  (summed  over  runs  -  not  average) 
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